Hallucination compounds in long-form generation because the divergence between correct and error-containing prefixes is not bounded by the training objective during teacher-forced training, allowing the divergence to grow arbitrarily as the generated prefix deviates from the true prefix.

Authors

• Person: M. Brenndoerfer Organization: mbrenndoerfer.com
  Hallucination Causes: Why Language Models Fabricate Facts

Sources

• Hallucination Causes: Why Language Models Fabricate Facts mbrenndoerfer.com M. Brenndoerfer · mbrenndoerfer.com via serper

Referenced by nodes (1)

• hallucination concept