The HalluMeasure procedure for measuring hallucinations consists of the following steps: (1) decompose the LLM response into a set of claims using a claim extraction model; (2) classify the claims into five classes (supported, absent, contradicted, partially supported, and unevaluatable) by comparing them to retrieved context; (3) classify the claims into 10 distinct linguistic-error types (e.g., entity, temporal, and overgeneralization); (4) calculate an aggregated hallucination score based on the rate of unsupported claims and the distribution of error types.

Authors

• Person: Not available Organization: Amazon Science
  Automating hallucination detection with chain-of-thought reasoning

Sources

• Automating hallucination detection with chain-of-thought reasoning www.amazon.science Amazon Science via serper

Referenced by nodes (1)

• context concept