Transformer-based models in NLP tasks commonly utilize the Adam optimizer and its variants, as documented in research by Vaswani et al. (2017b), Radford et al. (2019), and Brown et al. (2020).

Authors

• Person: Not available Organization: arXiv
  A Survey on the Theory and Mechanism of Large Language Models

Sources

• A Survey on the Theory and Mechanism of Large Language Models arxiv.org arXiv via serper

Referenced by nodes (2)

• natural language processing concept
• Transformer models concept