The log-probability of the correct answer at step t given the error context is: P(x_t | x_{1:t-1}, e_t) = P(x_t | x_{1:t-1}, e_t), where x_t is the correct token steps after the error, x_{1:t-1} are correctly generated tokens before the error occurred, and e_t are tokens generated after the error, each conditioned on the growing error context.

Authors

• Person: M. Brenndoerfer Organization: mbrenndoerfer.com
  Hallucination Causes: Why Language Models Fabricate Facts

Sources

• Hallucination Causes: Why Language Models Fabricate Facts mbrenndoerfer.com M. Brenndoerfer · mbrenndoerfer.com via serper

Referenced by nodes (1)

• Large Language Models concept