Yingqian Cui, Jie Ren, Pengfei He, Hui Liu, Jiliang Tang, and Yue Xing present a theoretical analysis comparing the exact convergence of single-head and multi-head attention in transformers for in-context learning with linear regression tasks.

Authors

• Person: Samuel Tesfazgi, Leonhard Sprandl, Sandra Hirche Organization: AISTATS
  Track: Poster Session 3 - aistats 2026

Sources

• Track: Poster Session 3 - aistats 2026 virtual.aistats.org Samuel Tesfazgi, Leonhard Sprandl, Sandra Hirche · AISTATS via serper

Referenced by nodes (2)

• Transformers concept
• In-Context Learning concept