Large language models are trained to always continue a sequence, and the training objective rewards producing probable continuations, which results in the absence of a built-in mechanism for stopping generation or outputting an 'uncertain' token when the model lacks knowledge.

Authors

• Person: M. Brenndoerfer Organization: mbrenndoerfer.com
  Hallucination Causes: Why Language Models Fabricate Facts

Sources

• Hallucination Causes: Why Language Models Fabricate Facts mbrenndoerfer.com M. Brenndoerfer · mbrenndoerfer.com via serper

Referenced by nodes (1)

• Large Language Models concept