The training objective for large language models is next-token prediction, where the model maximizes the log-probability of each correct next token given its context, with 'correct' defined as what appeared in the training corpus rather than what is factually true.

Authors

• Person: M. Brenndoerfer Organization: mbrenndoerfer.com
  Hallucination Causes: Why Language Models Fabricate Facts

Sources

• Hallucination Causes: Why Language Models Fabricate Facts mbrenndoerfer.com M. Brenndoerfer · mbrenndoerfer.com via serper

Referenced by nodes (1)

• Large Language Models concept