RAG pipelines, explained
for the non-engineer
Retrieval-Augmented Generation is the method most enterprises use to make AI actually useful on their own data. Understanding how it works — and where it breaks — is now a core due diligence skill.
Why general AI isn’t enough for enterprise use
A foundation model like GPT-4 or Claude is trained on public data up to a certain date. It knows a great deal about the world in general — and nothing specific about your company, your contracts, your clients, or your internal processes.
For most enterprise use cases — summarising deal memos, answering questions about internal policy, reviewing supplier contracts — that gap is fatal. A model that cannot access your data cannot do your work.
RAG is the dominant architectural solution to this problem. Rather than retraining a model on proprietary data (slow, expensive, and rarely necessary), RAG lets the model retrieve relevant information at the moment it is needed, then generate a response grounded in that information.
“RAG is how AI goes from impressive demo to operational tool. The gap between the two is where most enterprise AI projects quietly stall.”
How it works — without the engineering lecture
A RAG pipeline has three components. Each one is a potential point of failure — and a point of due diligence.
The four questions your board should be asking
Whether you are evaluating an acquisition target, governing a portfolio company, or steering your own transformation, these are the questions that separate informed oversight from misplaced confidence.
Poor document ingestion
If the indexing step is done carelessly — scanned PDFs not OCR'd, tables not parsed correctly, outdated documents mixed with current ones — the retrieval step will surface bad information. Garbage in, garbage out applies here with particular force.
Weak retrieval logic
Not all retrieval systems are equal. A poorly tuned system will return tangentially related passages rather than the most relevant ones, leading the model to generate plausible but incorrect answers. In a legal or financial context, this is a liability, not an inconvenience.
No source attribution
A well-built RAG system tells the user exactly which document and passage it drew from. Systems without this are opaque by design — the user cannot verify the answer, which undermines the core value proposition of grounding AI in real data.
Stale or uncontrolled data
The index is only as current as the data fed into it. If documents are not re-indexed when they are updated, the system will confidently retrieve — and the model will confidently cite — information that is no longer accurate. Data governance is a RAG problem, not just an IT problem.
What this means in a transaction
RAG is now the backbone of most enterprise AI products — knowledge management tools, AI-assisted legal review, internal copilots, customer-facing chatbots. When a target company claims its AI product is differentiated, the architecture is almost always RAG-based. The question is how well it is built.
In our due diligence work, four things determine whether a RAG implementation is a genuine asset or a fragile prototype: the quality of the document corpus, the sophistication of the retrieval layer, the presence of human review checkpoints, and the maturity of the data refresh cycle. A company that cannot describe all four clearly has not yet moved past the pilot stage.
Equally important: RAG systems process company documents through external model APIs. If a target has deployed RAG on sensitive data — client records, financial models, M&A materials — without adequate data processing agreements in place, that is a compliance exposure that belongs in the risk register, not a footnote.
The question that reveals RAG maturity
Ask any team running a RAG-based product one question: “What happens when the system retrieves the wrong document?”
Mature teams have a clear answer: evaluation frameworks that measure retrieval accuracy, feedback loops that flag incorrect responses, and defined escalation paths when confidence is low. Early-stage teams look surprised by the question. That response — the surprise — is the signal you needed.
