Retrieval augmented generation (RAG)

A technique that enhances generative AI outputs by retrieving relevant information from external knowledge sources and incorporating it into the generation process, reducing hallucination and improving accuracy.

Source: AIEOG AI Lexicon (Feb 2026), adapted from arXiv:2005.11401 and NIST AI 100-1

Retrieval augmented generation (RAG) is a technique that gives generative AI systems access to specific, authoritative information sources when generating responses. Instead of relying solely on what the model learned during training, RAG retrieves relevant documents or data from a knowledge base and includes them in the context the model uses to generate its answer.

Think of it this way: without RAG, a generative AI model is like an expert answering questions entirely from memory. With RAG, the model is like an expert who can look up information in a reference library before answering. The reference library makes the answers more accurate, more current, and more grounded in authoritative sources.

RAG has become the standard approach for enterprise AI applications where accuracy and currency matter. It addresses one of generative AI’s biggest weaknesses — hallucination — by providing the model with verified source material to draw from.

RAG is particularly valuable in financial services where accuracy and auditability are paramount:

• Regulatory compliance queries. RAG can ground AI responses in actual regulatory text, guidance documents, and enforcement actions rather than relying on the model’s potentially outdated or inaccurate training data.
• Policy and procedure lookups. Internal policy questions can be answered with reference to the institution’s actual policy documents.
• Customer service. Customer-facing AI can provide accurate product, fee, and account information by retrieving from verified sources.
• Audit and examination support. RAG-powered tools can cite specific documents and sections, creating a verifiable audit trail.

Governance considerations for RAG include knowledge base quality (the retrieved information must be accurate, current, and comprehensive), retrieval accuracy (the system must retrieve the right information for each query), and attribution (outputs should be traceable to their source documents).

1. Curate knowledge bases carefully. The quality of RAG outputs depends on the quality of the information retrieved. Ensure knowledge bases are accurate, current, and comprehensive.
2. Validate retrieval accuracy. Test that the system retrieves relevant information for a range of queries, including edge cases.
3. Require source attribution. Configure RAG systems to cite their sources, enabling verification and audit.
4. Monitor for hallucination despite RAG. RAG reduces but does not eliminate hallucination. Continue testing for accuracy.
5. Manage knowledge base updates. Establish processes to keep knowledge bases current as regulations, policies, and products change.
6. Include in AI governance. RAG implementations should be documented, including the knowledge sources, retrieval method, and validation results.

Generative AI, Large language model, Hallucination, Foundation models, Training data, Text/word embedding, Output validation