Inference in large language models computes the probability of the next token, denoted as P(y_hat_t | y_hat_<t), where y_hat_t is the token the model generates at step t and y_hat_<t represents the model's own previously generated tokens.

Authors

• Person: M. Brenndoerfer Organization: mbrenndoerfer.com
  Hallucination Causes: Why Language Models Fabricate Facts

Sources

• Hallucination Causes: Why Language Models Fabricate Facts mbrenndoerfer.com M. Brenndoerfer · mbrenndoerfer.com via serper

Referenced by nodes (2)

• Large Language Models concept
• inference concept