The mathematical consequence of exposure bias is that large language model probability estimates are well-calibrated for contexts matching the training distribution but poorly calibrated for out-of-distribution contexts that contain errors.

Authors

• Person: M. Brenndoerfer Organization: mbrenndoerfer.com
  Hallucination Causes: Why Language Models Fabricate Facts

Sources

• Hallucination Causes: Why Language Models Fabricate Facts mbrenndoerfer.com M. Brenndoerfer · mbrenndoerfer.com via serper

Referenced by nodes (1)

• exposure bias concept