What is RAG? Retrieval augmented generation, explained for business
Retrieval augmented generation is a way of making an AI answer from a specific set of documents rather than from everything it absorbed during training. When someone asks a question, the system first retrieves the most relevant passages from your material, then asks the model to answer using only those, and returns the answer with its sources attached.
What you need to know
- RAG makes an AI answer from your own documents, with a citation, instead of from general training data.
- It is the right choice when answers must be grounded in your material and traceable to a source.
- The hard part is never the AI. It is the state of your documents.
- A useful internal RAG system typically costs $10,000 to $25,000 to build, plus $50 to $200 a month to run.
By Daniellle Bhatt · Updated 13 August 2026 · Plain text
Where this comes from: We have built RAG systems in production, including an Airtable and Pinecone knowledge hub with an automated ingestion pipeline that processed around 360 documents in a single run with zero failures. The failure modes described here are ones we hit and had to design around.
Why it exists
A general model knows a great deal about the world and nothing about your business. It has never seen your pricing, your contracts, your product specifications or last quarter's board pack. Asked about them, it will either say it does not know, or produce something plausible and wrong.
RAG closes that gap without retraining anything. Your documents stay where they are; the system just gets very good at finding the right passage before answering.
How it works, in four steps
The citation step is what makes it usable in a business. An answer you can trace is an answer someone will act on.
- Ingest. Documents are split into passages and converted into vectors, which are numerical representations of meaning.
- Store. Those vectors go into a vector database such as Pinecone, or a database with vector support.
- Retrieve. A question is converted the same way, and the closest passages are pulled back. This is matching by meaning, not by keyword, so "what is our refund window" finds a passage about returns policy.
- Generate. The model is given those passages and asked to answer using them, citing which one it used.
What it is good and bad at
| Good at | Bad at |
|---|---|
| Answering from a defined body of documents | Questions needing a full-corpus view, like "summarise every contract" |
| Attaching a source to every answer | Arithmetic across many documents |
| Staying current, since you update documents not models | Anything where the documents are contradictory or out of date |
| Keeping proprietary material out of a public model | Judgement calls the documents do not cover |
The part that actually takes the time
Your documents. Every RAG project is a data project wearing a costume. Duplicate policies, three versions of the same price list, scanned PDFs with no text layer, and knowledge that only exists in somebody's head.
The system will faithfully retrieve the outdated price list if the outdated price list is what it was given. Garbage in, confidently cited garbage out, which is worse than no answer because it looks authoritative.
Budget real time for document cleanup. On most projects it is the largest single line item, and skipping it is the most reliable way to produce a system nobody trusts.
A RAG project is a data project wearing a costume. If your documents are a mess, the answer is to fix the documents, not to buy a better model.
What it costs
A useful internal system typically runs $10,000 to $25,000 to build, depending mainly on how much document preparation is required. Running costs are modest: $50 to $200 a month for the vector database and model usage at normal internal volumes.
Worth it when several people repeatedly ask the same questions of a large document set: support teams, sales teams answering technical questions, or any business where the answer exists somewhere and finding it takes twenty minutes.
Not worth it for a small document set. If it fits in a well-organised folder that ten people search occasionally, improve the folder. More on that decision in build or buy.
Sources
Want this applied to your business?
Thirty minutes. You describe how the business runs, we tell you which processes are worth automating and which are not. You will get a straight answer either way, and there is nothing attached to it.
Keep reading
Copilot vs ChatGPT: which one belongs in your business?
Copilot sits inside Word, Excel, Outlook and Teams. ChatGPT sits in a browser tab and does more. That difference decides it for most businesses, and not in the direction people expect.
Read it GuideWhat is Microsoft Copilot? A plain explanation for business
Copilot is not one product. It is a brand covering at least five different things, which is why the question is confusing. Here is what each one actually is and which you probably mean.
Read it