Instruction-following datasets used for supervised finetuning often have thin coverage of rare query types, meaning models receive little practice on the specific queries where they are most likely to hallucinate.

Authors

• Person: M. Brenndoerfer Organization: mbrenndoerfer.com
  Hallucination Causes: Why Language Models Fabricate Facts

Sources

• Hallucination Causes: Why Language Models Fabricate Facts mbrenndoerfer.com M. Brenndoerfer · mbrenndoerfer.com via serper

Referenced by nodes (2)

• Large Language Models concept
• supervised fine-tuning concept