Exposure bias causes hallucinations because teacher forcing creates a training-inference mismatch where the model is never trained to handle its own errors, causing early mistakes in generation to cascade across subsequent tokens.

Authors

• Person: M. Brenndoerfer Organization: mbrenndoerfer.com
  Hallucination Causes: Why Language Models Fabricate Facts

Sources

• Hallucination Causes: Why Language Models Fabricate Facts mbrenndoerfer.com M. Brenndoerfer · mbrenndoerfer.com via serper

Referenced by nodes (2)

• hallucination concept
• exposure bias concept