The custom hallucination detection system uses RAGAS (Retrieval Augmented Generation Automatic Score) metrics to evaluate LLM responses. If the hallucination score for a response falls below a custom threshold, the system notifies human agents via Amazon Simple Notification Service (Amazon SNS) to assist with the query instead of providing the customer with the hallucinated response.

Authors

• Person: Not available Organization: Amazon Web Services
  Reducing hallucinations in large language models with custom ...

Sources

• Reducing hallucinations in large language models with custom ... aws.amazon.com Amazon Web Services via serper

Referenced by nodes (1)

• RAGAS concept