Designing Your AI Application

Every course before this one taught you a piece: a model call, retrieval, tools, MCP, agents, memory. Each, on its own, felt like the obvious next step. The actual skill this course teaches is different and, I'd argue, harder: given a real problem, knowing which of those pieces you genuinely need — and, just as important, which ones you don't. The single most common mistake I see in AI system design isn't picking the wrong tool. It's reaching for every tool at once, on a project that needed one of them.

Before choosing anything, it helps to see the whole shape of what you're choosing between. Every AI application, however complex it looks from outside, is built from a small number of layers, added on top of each other only as far as the problem actually demands.

Click through all seven rungs before reading on, because the shape matters more than any individual rung: capability goes up steadily, but so does cost — and not evenly. Notice which rung has the widest cost range in the whole ladder. It's the agent loop, by a wide margin, and that's not a coincidence — every additional step in a loop is another full model call, and loops can run for a surprising, expensive number of steps if nothing bounds them. Keep that one fact in your head for the rest of this course; it comes up again and again.

The most common overbuild in this entire field happens before the first rung of the ladder is even reached: reaching for a language model when a much simpler, cheaper, more reliable piece of software would do the job.

A few honest questions worth asking first, in order:

Once you've confirmed you genuinely need a model, the next decision is how you get one into your product — and this decision has real, ongoing cost consequences, not just a one-time setup choice.

A model with the right general reasoning still doesn't know your private, current, or specific facts. Three genuinely different tools solve that, and they aren't interchangeable.

RAG (see Production RAG for the full depth) retrieves the relevant documents at request time and hands them to the model as part of the prompt. It's the right default for most knowledge-grounding needs: it stays current automatically as your documents change, and it can cite exactly what it drew from.

Long context — simply pasting a large amount of material directly into the prompt every time, no retrieval step — works for a genuinely small, fixed set of documents that fit comfortably in the context window. It's simpler than RAG, but it doesn't scale past that window, and it re-sends (and re-pays for) the same material on every single request.

Fine-tuning teaches the model a pattern or a style through training examples, not through facts handed to it at request time. It's the wrong tool for "know these specific facts" — facts baked in through fine-tuning still go stale, and updating them means retraining. It's the right tool for "always respond in this specific format" or "always apply this house style," where the pattern matters more than any single fact.

An agent — see The ReAct Loop — lets a model plan, act, observe, and repeat across multiple steps toward a goal, instead of answering in one shot. It's genuinely powerful, and it's also the layer most responsible for AI projects quietly running over budget.

The reason is simple arithmetic that's easy to lose sight of while building: every loop iteration is another full model call. A task that takes five iterations to finish costs roughly five times a single call — and if something goes wrong and the loop doesn't recognise it's finished, that multiplier keeps climbing. This is precisely why Loop Engineering — designing the loop so it reliably converges instead of spinning — is its own dedicated course in this track.

Before reaching for an agent loop at all, ask honestly: does this task reliably finish in one or two calls? Most tasks — genuinely most — do. An agent loop is the right tool for tasks whose shape you can't fully predict in advance (multi-step research, exploratory debugging), not a default upgrade for every feature.

Once a model needs to actually do something — not just describe what should happen — it needs tools: defined actions it can call, with real effects on the outside world (checking a database, sending an email, updating a record).

You can wire tools up two ways. Hand-rolled function calling means you write a custom integration for every tool, specific to your codebase — fine for one or two simple, stable tools. MCP (Model Context Protocol — see MCP for Practical Builders) is a standard way to expose tools that any compatible model or agent can use, without a bespoke integration for each one.

The cost story here is different from every other rung: MCP itself doesn't meaningfully change your per-request token cost. What it changes is integration and maintenance effort — the real cost it saves is engineering time, not inference spend.

There's a layer above a single tool call that's worth understanding on its own: a Skill — a packaged, reusable bundle of instructions, reference material, and sometimes scripts, that teaches a model how to do a specific, repeatable job well, rather than giving it one narrow action to call.

The difference in practice: a tool answers "what can it do." A Skill answers "how should it do this particular kind of job, every time, the way an experienced person on your team would." Without a Skill, a model has to work out the right process from scratch, from your prompt alone, every single time you ask for something like a quarterly report. With one, the expertise for that specific, repeatable job is captured once and reused — closer to onboarding a new team member with a well-written playbook than to just handing them a tool.

Giving AI a Memory covers this in full depth — the four kinds (working, episodic, semantic, procedural) and the actual tools (claude-mem, mem0, Letta, checkpointers). Here, the question is simpler and comes first: do you need any of it at all?

A huge share of AI applications genuinely don't. If every session is independent — a one-off question, a single document summarised and done — adding a memory layer is pure cost and complexity with nothing to show for it. Memory earns its cost specifically when a user's history should change how the system behaves next time: a returning customer whose past issues matter, an agent whose plan should build on yesterday's partially-finished work.

Theory earns its keep once it meets a real brief. Here's the same product idea, taken seriously at two genuinely different scales — because "the right stack" isn't one answer, it's a function of budget, volume, and what's actually at stake if something goes wrong.

Notice what actually changes between the two columns: it's not that the enterprise version uses "better" technology throughout. Several choices — RAG over fine-tuning, a bounded agent loop — are identical in both. What genuinely changes is how much risk gets engineered out: access control, audit trails, step limits with real teeth, isolated memory per customer. The bootstrapped version isn't a worse stack. It's a stack correctly sized to what a small team can actually operate and afford, with the discipline to add more only once real usage demands it.

Everything in this course compresses into one practical tool: answer honestly, and see what you actually need.