Knowledge Tree
Amazon Web Services
Also known as: AWS
Facts (30)
Sources
Detect hallucinations for RAG-based systems - AWS aws.amazon.com May 16, 2025 6 facts
claimThe AWS hallucination detection method requires that the LLM, memory, and parameters used for evaluation be independent of the ones used for the original Q&A to prevent the evaluation LLM from accessing previous chat history.
referenceThe AWS blog post identifies four prominent approaches for detecting hallucinations in RAG-based systems: using an LLM prompt-based detector, a semantic similarity detector, a BERT stochastic checker, and a token similarity detector.
procedureTo implement the hallucination detection methods described in the AWS blog post, a user requires an AWS account with access to Amazon SageMaker, Amazon Bedrock, and Amazon Simple Storage Service (Amazon S3).
procedureThe AWS approach for detecting hallucinations in RAG-based systems involves the following steps: 1) Create a dataset with questions, context, and the response to classify. 2) Send a call to an LLM providing the statement and the context. 3) Instruct the LLM to tag sentences in the statement that are directly based on the context. 4) Parse the outputs to obtain sentence-level numeric scores between 0 and 1. 5) Ensure the LLM, memory, and parameters used for evaluation are independent of those used for Q&A. 6) Tune the decision threshold for the hallucination scores based on the specific dataset or domain. 7) Use the threshold to classify the statement as a hallucination or fact.
codeThe configure_llm function in the AWS blog post configures an Amazon Bedrock client using the anthropic.claude-3-5-sonnet-20240620-v1:0 model with a temperature setting of 0.0 and specific stop words to prevent further generation.
claimIn the AWS hallucination detection method, the hallucination score is a float between 0 and 1, where 0 indicates the sentence is directly based on the context and 1 indicates the sentence has no basis in the context.
Evaluating RAG applications with Amazon Bedrock knowledge base ... aws.amazon.com Mar 14, 2025 5 facts
accountIshan Singh is a Generative AI Data Scientist at Amazon Web Services who specializes in building generative AI solutions.
procedureTo use the Amazon Bedrock knowledge base evaluation feature, users must have an active AWS account, enabled evaluator and generator models in Amazon Bedrock, confirmed AWS Region availability and quotas, configured AWS Identity and Access Management (IAM) permissions for an S3 bucket, enabled CORS on the S3 bucket, and created an Amazon Bedrock knowledge base with synced data.
accountJesse Manders is a Senior Product Manager on Amazon Bedrock, the AWS Generative AI developer service, with an academic background including an M.S. and Ph.D. from the University of Florida and an MBA from the University of California, Berkeley, Haas School of Business.
accountAyan Ray is a Senior Generative AI Partner Solutions Architect at Amazon Web Services with over a decade of experience in Artificial Intelligence and Machine Learning.
referenceAmazon Web Services provides a Jupyter notebook containing practical examples and code snippets for Amazon Bedrock Evaluations on their GitHub repository.
Knowledge graphs - Amazon Science amazon.science 4 facts
claimAmazon Web Services (AWS) hosts employee-led affinity groups to foster an inclusive team culture.
claimAmazon Connect is a cloud-based contact center service from AWS that enables businesses to deliver intelligent, engaging, dynamic, and personalized customer service experiences.
claimAmazon Web Services (AWS) hosts the Conversations on Race and Ethnicity (CORE) and AmazeCon conferences to support gender diversity and inclusion.
claimAmazon Web Services (AWS) is a cloud platform that provides a suite of products and services to startups and Global 500 companies.
bureado/awesome-software-supply-chain-security - GitHub github.com 2 facts
referenceThe mongodb/kingfisher tool is a high-performance secret scanner that validates secrets by testing them against external systems, utilizing entropy analysis and real-time verification across AWS, Azure, GCP, and other platforms to reduce false positives.
referenceAWS UEFI firmware for Nitro instances supports reproducible builds, allowing users to verify measurements against source code to enable reproducible hardware attestation.
The Role of Hallucinations in Large Language Models - CloudThat cloudthat.com Sep 1, 2025 1 fact
claimCloudThat is an award-winning company and the first in India to offer cloud training and consulting services worldwide, serving as a Microsoft Solutions Partner, AWS Advanced Tier Training Partner, and Google Cloud Platform Partner.
Reducing hallucinations in large language models with custom ... aws.amazon.com Nov 26, 2024 1 fact
accountShayan Ray is an Applied Scientist at Amazon Web Services whose research focuses on natural language processing, natural language understanding, natural language generation, conversational AI, task-oriented dialogue systems, and LLM-based agents.
How Neuro-Symbolic AI Breaks the Limits of LLMs - WIRED wired.com 1 fact
quote"The future of trustworthy AI agents starts with automated reasoning," says Cook, who is also the founder of the Automated Reasoning Group at AWS.
Business ecosystems as a way to activate lock-in in business models link.springer.com Mar 28, 2025 1 fact
claimAmazon established the Amazon Web Services business line to increase the efficiency of its consumer business by providing cheaper technology capabilities.
Archetypes of open-source business models | Electronic Markets link.springer.com Jun 14, 2022 1 fact
referenceThe article 'It’s not OK: Elastic takes aim at AWS, at the risk of major collateral damage' by Tom Krazit (2021) discusses the conflict between Elastic and Amazon Web Services (AWS) regarding open source licensing and business strategy.
Empowering RAG Using Knowledge Graphs: KG+RAG = G-RAG neurons-lab.com 1 fact
claimNeurons Lab is an AWS Advanced Partner and holds a GenAI competency.
What is the impact of open-source on the tech industry? - Milvus milvus.io 1 fact
claimElastic and MongoDB changed their software licenses to prevent Amazon Web Services from offering managed versions of their databases without reciprocity, highlighting the tension between monetization and open-source sustainability.
LLM Hallucination Detection and Mitigation: State of the Art in 2026 zylos.ai Jan 27, 2026 1 fact
referenceAWS provides documentation titled 'Detect hallucinations for RAG-based systems,' which outlines strategies for identifying hallucinations in retrieval-augmented generation (RAG) architectures.
Role of Open Source Software in Rise of AI nutanix.com 1 fact
measurementMajor cloud providers, including Microsoft and Amazon Web Services, have embraced Red Hat OpenShift within the last five years.
Weekly Innovations and Future Trends in Open Source dev.to May 19, 2025 1 fact
claimKubernetes 1.32 provides enhanced multi-cloud support through integration with AWS, Azure, and GCP, while introducing zero-trust security policies.
Cybersecurity Trends and Predictions 2025 From Industry Insiders itprotoday.com 1 fact
perspectiveAnn Irvine does not believe that Amazon Web Services (AWS) will experience an outage lasting more than 24 hours in 2025.
Open-source license - Wikipedia en.wikipedia.org 1 fact
perspectiveAmazon Web Services has stated that they comply with open-source licenses and act in their customers' best interests.
Context Graph vs Knowledge Graph: Key Differences for AI - Atlan atlan.com Jan 27, 2026 1 fact
claimThe Model Context Protocol (MCP) is a Linux Foundation project supported by AWS, Anthropic, Google, Microsoft, and OpenAI.