Large language models experience a training-inference mismatch because the conditioning context during training is always perfect ground-truth tokens, whereas the conditioning context during inference is the model's own outputs, which may contain errors.

Authors

• Person: M. Brenndoerfer Organization: mbrenndoerfer.com
  Hallucination Causes: Why Language Models Fabricate Facts

Sources

• Hallucination Causes: Why Language Models Fabricate Facts mbrenndoerfer.com M. Brenndoerfer · mbrenndoerfer.com via serper

Referenced by nodes (1)

• Large Language Models concept