Knowledge Tree
knowledge gaps
Facts (11)
Sources
Hallucination Causes: Why Language Models Fabricate Facts mbrenndoerfer.com Mar 15, 2026 5 facts
claimKnowledge gaps cause hallucinations because training cutoffs, tail entity under-representation, restricted access to specialized domains, and the absence of a symbolic world model mean that many factual questions fall outside the model's reliable knowledge boundary, yet the model cannot reliably identify when it is operating outside that boundary.
claimRobust approaches to mitigating large language model hallucinations target multiple causes simultaneously, including using retrieval augmentation for knowledge gaps, better data curation for training data issues, scheduled sampling variants for exposure bias, and calibration training for generation pressure.
claimInstruction tuning and reinforcement learning from human feedback (RLHF) improve a large language model's ability to express uncertainty and abstain from answering when knowledge is insufficient, but they do not retroactively fill knowledge gaps or undo exposure bias present in the base model.
claimTraining data issues, exposure bias, knowledge gaps, and generation pressure are recognized phenomena contributing to large language model hallucinations, but quantifying their individual contributions to specific hallucinations is difficult.
claimFinetuning large language models modifies the model's response style regarding expressed confidence, but the underlying knowledge gaps and exposure bias patterns remain encoded in the base model from pretraining.
LLM Hallucinations: Causes, Consequences, Prevention - LLMs llmmodels.org May 10, 2024 2 facts
referenceThe causes of LLM hallucinations include flawed training data (biases, inaccuracies, or inconsistencies), knowledge gaps (lack of domain-specific knowledge or context understanding), and technical limitations (over-reliance on statistical patterns and vulnerability to manipulation).
claimLarge language models can hallucinate due to knowledge gaps and context issues, as they may not always understand the context in which text is being used despite processing vast amounts of data.
Medical Hallucination in Foundation Models and Their ... medrxiv.org Mar 3, 2025 2 facts
claimInadequate training data coverage creates knowledge gaps that cause large language models to hallucinate when addressing unfamiliar medical topics, according to Lee et al. (2024).
referenceHou et al. (2024) developed a semantic entropy-based method that analyzes how an AI model responds to different versions of the same question to distinguish between uncertainty caused by unclear question phrasings and uncertainty due to the model’s own knowledge gaps.
Practices, opportunities and challenges in the fusion of knowledge ... frontiersin.org 1 fact
claimKnowledge Tracing empowered by knowledge graphs allows large language models (LLMs) to track knowledge evolution, fill in knowledge gaps, and improve the accuracy of responses.
A Mixed-Methods Study of Open-Source Software Maintainers On ... arxiv.org Feb 3, 2025 1 fact
claimFactors contributing to a lack of understanding in Open Source Software vulnerability management include system complexity, the difficulty of developing a patch, testing requirements, and knowledge gaps.