Wang et al. (2023) identified that in input-label pairs during in-context learning (ICL), label tokens act as anchors where semantic information from the context aggregates at the shallower layers of large language models, and final predictions reference this aggregated information.

Authors

• Person: Not available Organization: arXiv
  A Survey on the Theory and Mechanism of Large Language Models

Sources

• A Survey on the Theory and Mechanism of Large Language Models arxiv.org arXiv via serper

Referenced by nodes (2)

• Large Language Models concept
• In-Context Learning concept