Exposure bias in large language models creates compounding errors where a small factual inaccuracy or semantic drift at a specific position changes the conditioning context for subsequent positions, leading the model to generate statistically likely continuations based on erroneous premises.

Authors

• Person: M. Brenndoerfer Organization: mbrenndoerfer.com
  Hallucination Causes: Why Language Models Fabricate Facts

Sources

• Hallucination Causes: Why Language Models Fabricate Facts mbrenndoerfer.com M. Brenndoerfer · mbrenndoerfer.com via serper

Referenced by nodes (2)

• Large Language Models concept
• exposure bias concept