Exposure bias accumulation occurs when a model generates an incorrect token, which then becomes part of the conditioning context for all subsequent tokens, causing the model to amplify details consistent with the error and suppress details associated with the correct information.

Authors

• Person: M. Brenndoerfer Organization: mbrenndoerfer.com
  Hallucination Causes: Why Language Models Fabricate Facts

Sources

• Hallucination Causes: Why Language Models Fabricate Facts mbrenndoerfer.com M. Brenndoerfer · mbrenndoerfer.com via serper

Referenced by nodes (1)

• exposure bias concept