What Is RAG?

By Saad Sahi ·

Short answer: RAG (retrieval-augmented generation) is how an AI assistant answers from your company's own documents instead of guessing from its training data. When someone asks a question, the system pulls the relevant passages from your files and writes the answer from those passages — with sources you can check. It's the technology behind private knowledge assistants for handbooks, policies, and procedures.

How RAG works, step by step

  1. Your documents are indexed. Handbooks, policies, manuals, price lists — whatever your team asks about repeatedly — are converted into a searchable index. The originals stay where they are.
  2. Someone asks a question. "What's our overtime policy for weekend shifts?"
  3. The system retrieves the relevant passages. It searches the index and pulls the few paragraphs most likely to contain the answer.
  4. The AI writes an answer from those passages. It doesn't invent from training data; it composes from what it just retrieved, and shows which documents it used so anyone can verify.

The key idea: the AI's knowledge comes from your files at the moment of the question, not from whatever it learned during training. Update a document, and the assistant's answers update with it — no retraining needed.

What a private knowledge assistant looks like in practice

  • Policy and handbook assistant. New hires ask questions in plain language instead of digging through a 60-page PDF. "How many sick days do I get?" gets an answer with a link to the exact section.
  • Pricing and quoting assistant. Sales staff ask "what's our rate for X with Y options?" and get the current answer from the price book, not last quarter's memory.
  • Support documentation assistant. Technicians in the field ask how to handle a specific fault and get the procedure from the manual, step by step.
  • Client-facing FAQ assistant. Common customer questions get accurate answers drawn from your real documentation, with a handoff to a human when the question goes beyond it.

AI Alberta builds private AI assistants like these for Alberta businesses.

Privacy, honestly

RAG on its own doesn't automatically make anything private — privacy comes from how the system is built and where it runs. Here's what actually matters:

  • Access controls. The assistant should only answer from documents the person asking is allowed to see. A shop-floor employee shouldn't get payroll policy details just because they're in the same index.
  • Where it runs. Your documents stay in your systems and accounts. A properly built assistant doesn't ship your files off to places you don't control.
  • No training on your data. Your business documents are used to answer questions, not to train public AI models. These are separate things, and any vendor should be able to explain the difference clearly.
  • Auditability. Because answers cite their sources, you can always check what the assistant based an answer on — which is more than you get from an employee's best recollection.

If a vendor can't explain these four points in plain language, that's a red flag regardless of whose logo is on the proposal.

When RAG is the right tool

RAG fits when the answers live in documents that change: policies, price lists, manuals, procedures. If your documents rarely change and the questions are simple, a well-organised FAQ or search might do the job for less money. RAG earns its keep where people currently interrupt colleagues to ask things that are written down somewhere.

Questions

Frequently asked questions

Is RAG the same as training an AI on my data?+

No. Training bakes knowledge into the model permanently — expensive, slow to update, and hard to audit. RAG looks up your documents at question time, so answers stay current and always show their sources. For most businesses, RAG is the simpler and more practical choice.

How many documents do we need for RAG to be useful?+

Fewer than you'd think. A single handbook, price book, or manual that people ask about daily is enough to start. Begin with the documents behind your most repeated questions and expand from there. Quality matters more than quantity — a small set of current, well-kept documents beats a huge archive of outdated files.

Can the assistant answer questions it shouldn't?+

It shouldn't, if access controls are set up properly. Permissions mirror your existing document permissions: people only get answers from files they could already open. This is a design requirement, not an optional extra — confirm it during scoping, and test it with restricted documents before the assistant goes live.

How much does a private knowledge assistant cost?+

It depends on the number of documents, how they're stored, and how the assistant is delivered (internal tool, website widget, etc.). AI Alberta's automation projects start from $1,500 CAD with a fixed quote agreed before work begins. See our pricing page for starting prices.

Discuss your AI project

If your team answers the same questions from the same documents every week, a private knowledge assistant could save real hours. Book an AI Automation Consultation or call +1 587-987-0039 — describe the documents and the questions, and you'll get a plain-English plan and a fixed price before committing to anything.