Knowledge Tree
Relations (1)
related 2.32 — strongly supporting 4 facts
LLM-as-a-judge is a method used to evaluate factual correctness, showing closer alignment with human assessments than lexical metrics like ROUGE [1], [2]. However, research indicates that these evaluation methods may still struggle with genuine factual correctness when compared to human benchmarks [3], [4].
Facts (4)
Sources
Re-evaluating Hallucination Detection in LLMs - arXiv arxiv.org 4 facts
claimLLM-as-Judge evaluation, when validated against human judgments, reveals significant performance drops across all hallucination detection methods when they are assessed based on factual accuracy.
claimAn evaluation method based on 'LLM-as-Judge' demonstrates closer agreement with human assessments of factual correctness compared to ROUGE, according to Thakur et al. (2025).
claimThe moderate Pearson correlation coefficient between AUROC scores derived from ROUGE and LLM-as-Judge evaluation approaches suggests that hallucination detection methods may be inadvertently optimized for ROUGE’s lexical overlap criteria rather than genuine factual correctness.
claimThe 'LLM-as-Judge' evaluation method offers a closer alignment with human judgments of factual correctness compared to ROUGE, as validated by the human study conducted by the authors of 'Re-evaluating Hallucination Detection in LLMs'.