The procedure to build the Vectara hallucination leaderboard involves: (1) feeding the full set of documents in the dataset to each Large Language Model, (2) asking the models to summarize each document using only the facts presented in the document, (3) computing the overall factual consistency rate (no hallucinations) and hallucination rate (100 minus accuracy) for each model, and (4) recording the rate at which each model refuses to respond in an 'Answer Rate' column.

Authors

• Person: Not available Organization: Vectara
  vectara/hallucination-leaderboard - GitHub

Sources

• vectara/hallucination-leaderboard - GitHub github.com Vectara via serper

Referenced by nodes (1)

• Vectara entity