An LLM on its own is a brilliant improviser with a frozen, fuzzy memory. It does not know your company's documents, today's date, or the answer to "what's 8,431 × 297." This course is about fixing that — giving the model real knowledge (RAG) and real abilities (tools). This is the step that turns a clever chatbot into a system you can actually rely on.
- Understand why a raw LLM cannot be trusted with facts — and how RAG fixes it
- Follow the RAG pipeline end to end, in plain steps
- Understand tool use: letting the model search, calculate, and call APIs
- Know when to retrieve, when to use a tool, and when to just answer
Why a model needs the outside world
From the last course: an LLM predicts plausible text from a frozen snapshot of what it read during training. That creates three hard limits:
- It is out of date. It knows nothing after its training cut-off — not today's news, not your latest policy.
- It does not know your private data. Your company wiki, your customer records, your codebase — invisible to it.
- It guesses at facts. Ask for a specific number or citation and it may confidently invent one.
You cannot retrain a giant model every time a document changes — that is absurdly expensive. The smart move is to leave the model as-is and hand it the right information at question time. That is the whole idea behind RAG.
- A raw LLM is out of date, blind to your private data, and prone to inventing facts
- Retraining for every new document is impractical
- The fix: keep the model fixed and supply the right information at question time
RAG, step by step
RAG stands for Retrieval-Augmented Generation. Despite the name, the idea is simple: find the relevant facts, then let the model answer using them.
The clever part is step 1 — finding the right passages. This uses the "things become points in space" idea from the Maths course: the question and every document chunk are turned into vectors, and the system grabs the chunks whose vectors sit closest to the question's. (That technique, semantic search, has its own hands-on course in Applied AI.)
- RAG = search your documents, retrieve the best passages, answer from them
- Finding the right passages uses semantic (meaning-based) search over vectors
- Answers become grounded, current, and citable instead of guessed
- Why is RAG better than retraining the model every time a document changes?
- What does the "retrieve" step actually look for, and how does it find it?
Tools — letting the model act
RAG lets the model read. Tools let it do. A tool is any capability you hand the model: a calculator, a web search, a database lookup, sending an email, running code.
The model does not run the tool itself. Instead, when it realises it needs one, it outputs a structured request — "call the calculator with 8431 × 297" — your system runs it, and the result is handed back to the model to continue. This is often called function calling.
User: "What's 8431 × 297, and is it more than our budget of 2.4M?"
|
v
Model: needs maths → asks to call calculator(8431 * 297)
|
System runs it → 2,503, ...wait, 2,503, 207
|
v
Model: reads result → "That's 2,503,207 — yes, over your 2.4M budget."
This is how an AI assistant can check live weather, look up an order, book a meeting, or run a piece of code. The model supplies the reasoning and language; the tools supply truth and action.
- A tool is any capability the model can call: calculator, search, database, API, code
- The model requests a tool; your system runs it; the result returns to the model
- Tools provide the exactness and real-world actions LLMs lack
When to retrieve, when to use a tool, when to just answer
A good AI system does not RAG or call tools for everything — that would be slow and wasteful. Part of the craft is choosing:
| The question needs… | Use | Example |
|---|---|---|
| Facts from your documents | RAG | "What's our refund policy?" |
| A live value or an action | A tool | "What's the weather in Tokyo?" / "Book the room." |
| Exact calculation | A tool | "What's 18% of 4,317?" |
| General knowledge or reasoning | Just answer | "Explain why the sky is blue." |
Often a single request mixes these — "Summarise our Q3 results [RAG] and email them to the team [tool]." Orchestrating that gracefully is where the next courses (Context, Harness, and Loop Engineering) come in. This course gives you the two building blocks; the rest of the track assembles them into reliable systems.
- Don't retrieve or call tools for everything — choose based on what the question needs
- RAG for your documents, tools for live data/actions/maths, plain answers for general reasoning
- Combining them well is the foundation of capable AI agents
- Give one question best served by RAG and one best served by a tool.
- Why shouldn't an AI system use RAG or tools for every single question?