inference ↔ Large Language Models

Relations (1)

related 0.30 — supporting 3 facts

Inference is a fundamental stage in the lifecycle of Large Language Models as defined in [1], and it serves as the core mechanism for generating token probabilities as described in [2]. Furthermore, techniques like DynaThink are specifically designed to optimize this inference process for Large Language Models, as noted in [3].

Facts (3)

Sources

Hallucination Causes: Why Language Models Fabricate Facts mbrenndoerfer.com M. Brenndoerfer · mbrenndoerfer.com 1 fact

formulaInference in large language models computes the probability of the next token, denoted as P(y_hat_t | y_hat_<t), where y_hat_t is the token the model generates at step t and y_hat_<t represents the model's own previously generated tokens.

A Survey of Incorporating Psychological Theories in LLMs - arXiv arxiv.org arXiv 1 fact

claimDynaThink is a technique that dynamically selects between rapid or thorough inference for Large Language Models.

A Survey on the Theory and Mechanism of Large Language Models arxiv.org arXiv 1 fact

claimThe survey titled 'A Survey on the Theory and Mechanism of Large Language Models' organizes the theoretical landscape of Large Language Models into a lifecycle-based taxonomy consisting of six stages: Data Preparation, Model Preparation, Training, Alignment, Inference, and Evaluation.