Precision

recall

Justification not yet generated — showing supporting facts

• F1 Score considers both the precision and recall of predicted answers, providing a balance between the two metrics.
• F1-Score is calculated as 2 multiplied by the quotient of (Precision * Recall) divided by (Precision + Recall).
• Hallucination detection methods often utilize metrics such as Recall, Precision, and K-Precision to evaluate the performance of the detector.
• The BERT score has been shown to correlate with human judgment on both sentence-level and system-level evaluation and computes precision, recall, and F1 measures for language generation tasks.
• The paper 'A Stitch in Time Saves Nine: Detecting and Mitigating Hallucinations of LLMs by Validating Low-Confidence Generation' uses Precision and Recall metrics to detect sentence-level and concept-level hallucinations in ChatGPT-generated paragraphs spanning 150 topics.
• For use cases where precision is the highest priority, the token similarity, LLM prompt-based, and semantic similarity methods are recommended, whereas the BERT stochastic method outperforms other methods for high recall.
• The AutoAIS (Attributable to Identified Sources) evaluation framework for AI systems utilizes zero-shot and fine-tuned precision, recall, and F1 metrics, as well as FActScore F1 scores based on reference factuality labels.
• The Q² metric evaluates factual consistency in knowledge-grounded dialogues and is compared against F1 token-level overlap, Precision and Recall, Q² w/o NLI, E2E NLI, Overlap, BERTScore, and BLEU using the WoW, Topical-Chat, and Dialogue NLI datasets.
• Evaluation metrics for hallucination detection include Precision, Recall, and F1, while metrics for mitigation include the ratio of self-contradiction removed, the ratio of informative facts retained, and perplexity increase.
• The BERTScore evaluation method proceeds in four steps: (1) map the words of the generated text and reference text to the embedding space to obtain word vectors, (2) calculate the cosine similarity for each word pair between the generated and reference texts to form a similarity matrix, (3) calculate Precision (P) as the average similarity of each word vector in the generated text to the most similar word vector in the reference text, and (4) calculate Recall (R) as the average similarity of each word vector in the reference text to the most similar word vector in the generated text.
• The evaluation of the LLM-knowledge graph framework demonstrated high performance in metrics including NDCG, precision, recall, and user satisfaction, with notable improvements in prioritization accuracy and expert identification.
• F1-Score is a measure that combines precision and recall, calculated as the harmonic mean of precision and recall, used in binary and multi-class classification tasks to provide a balanced evaluation of model performance.
• The LARS uncertainty estimation technique is evaluated using Accuracy, Precision, Recall, and AUROC metrics on the TriviaQA, GSM8k, SVAMP, and Common-sense QA datasets.
• Evaluation metrics for hallucination detection include Accuracy (Acc), G-Mean, BSS, AUC, and Precision, Recall, and F1 scores for both 'Not Hallucination' and 'Hallucination' classifications.
• The Directional Levy/Holt metric uses precision and recall with entity insertions and replacements to evaluate models on the Levy/Holt dataset, which consists of premise-hypothesis pairs.
• Evaluation metrics for AI systems include Precision, Recall, and F1 scores calculated under cross-examination strategies such as AYS, IDK, Confidence-Based, and IC-IDK.
• Classical metrics, including Precision, Recall, Accuracy, and F1-score, are used to quantify performance in the study.
• A custom fine-grained hallucination detection dataset categorizes factual hallucinations into Entity, Relation, Contradictory, Invented, Subjective, and Unverifiable types, evaluated using Precision, Recall, and F1 metrics.
• The ClaimDecomp dataset contains 1200 complex claims from PolitiFact, each labeled with one of six veracity labels, a justification paragraph from expert fact-checkers, and subquestions annotated by prior work, evaluated using accuracy, F1, precision, and recall.
• F1 Score is calculated based on precision and recall, where precision measures the correctness of retrieved information and recall assesses the proportion of target data retrieved.
• Evaluation metrics for list-based questions on Wikidata and Wiki-Category List include test precision and the average number of positive and negative hallucination entities; MultiSpanQA uses F1, Precision, and Recall; and longform generation of biographies uses FactScore.
• A white-box hallucination detector approach treats the Large Language Model as a dynamic graph and analyzes structural properties of internal attention mechanisms. This method extracts spectral features, specifically eigenvalues, from attention maps to predict fabrication: factual retrieval produces stable eigen-structures, while hallucination leads to diffuse, chaotic patterns. This detector operates independently of generated semantic content and was evaluated across seven QA benchmarks (NQ-Open, TriviaQA, CoQA, SQuADv2, HaluEval-QA, TruthfulQA, GSM8K) using AUROC, Precision, Recall, and Cohen's Kappa metrics.
• Cleanlab evaluates popular hallucination detectors across four public Retrieval-Augmented Generation (RAG) datasets using precision and recall metrics.
• Evaluation metrics for Large Language Models integrated with Knowledge Graphs vary depending on the specific downstream tasks and can include accuracy, F1-score, precision, and recall.
• Evaluation benchmarks for vision-language hallucination detection and mitigation include MHaluBench, MFHaluBench, Object HalBench, AMBER, MMHal-Bench, and POPE, which utilize metrics such as accuracy, precision, recall, F1-score, CHAIR, Cover, Hal, and Cog.
• QuestEval, proposed by Scialom et al. (2021), combines recall and precision by generating questions from both the source and the summary, and introduces a weighting mechanism for key information to improve evaluation robustness.
• Evaluation metrics for estimating hallucination degree include BLEU, ROUGE-L, FeQA, QuestEval, and EntityCoverage (Precision, Recall, F1).

accuracy

Justification not yet generated — showing supporting facts

• The LARS uncertainty estimation technique is evaluated using Accuracy, Precision, Recall, and AUROC metrics on the TriviaQA, GSM8k, SVAMP, and Common-sense QA datasets.
• Evaluation metrics for hallucination detection include Accuracy (Acc), G-Mean, BSS, AUC, and Precision, Recall, and F1 scores for both 'Not Hallucination' and 'Hallucination' classifications.
• To evaluate the KG-RAG approach against vector RAG and no-RAG baselines, the researchers incorporated a conventional accuracy metric and introduced a modified precision metric designed to quantify the incidence of hallucinations.
• Classical metrics, including Precision, Recall, Accuracy, and F1-score, are used to quantify performance in the study.
• The ClaimDecomp dataset contains 1200 complex claims from PolitiFact, each labeled with one of six veracity labels, a justification paragraph from expert fact-checkers, and subquestions annotated by prior work, evaluated using accuracy, F1, precision, and recall.
• Evaluation metrics for Large Language Models integrated with Knowledge Graphs vary depending on the specific downstream tasks and can include accuracy, F1-score, precision, and recall.
• Evaluation benchmarks for vision-language hallucination detection and mitigation include MHaluBench, MFHaluBench, Object HalBench, AMBER, MMHal-Bench, and POPE, which utilize metrics such as accuracy, precision, recall, F1-score, CHAIR, Cover, Hal, and Cog.

hallucination

The concepts are related because a modified precision metric was specifically introduced to quantify the incidence of hallucinations in the KG-RAG evaluation framework [1], and both are categorized as key evaluation metrics for Large Language Models in question answering tasks [2]. — 2 supporting facts

hallucination detection

Precision is a key performance metric used to evaluate the effectiveness of hallucination detection methods, as established in [1]. — 1 supporting fact

Large Language Models

Precision is identified as a key evaluation metric used to assess the performance of Large Language Models when integrated with Knowledge Graphs for specific downstream tasks, as stated in [1]. — 1 supporting fact

knowledge graphs

Knowledge graphs are related to precision because precision is identified as one of the evaluation metrics used to assess the performance of Large Language Models when they are integrated with knowledge graphs, as stated in [1]. — 1 supporting fact