Giving AI a Memory

"Does it have memory?" is one of the first questions anyone asks about an AI product, right after "is it smart?" — and it's the more interesting question, because most of what feels like intelligence in a good AI system is actually memory wearing a trench coat. A model that remembers your preferences looks thoughtful. A model that remembers nothing looks the same on every single call, no matter how capable it is underneath.

This course is a working architect's tour of that problem: what "memory" is actually made of, the tools people reach for in 2026 to build it — claude-mem, mem0, Letta, LangGraph's own checkpointer, the memory baked into ChatGPT and Claude themselves — and, more usefully, when each one is the right call and when it's the wrong one. No hype, no single "best" answer. Just the trade-offs, the way you'd explain them to a colleague over coffee before a design review.

Step back before we get technical. There's a clean line running through the last few years of AI products, and memory sits right at the hinge of it.

Plain software never needed to "remember" in any interesting sense — a database row is memory, but nobody calls it that, because it's just... state. Then came the AI copilot era: genuinely useful, and genuinely goldfish-brained. Every chat starts from zero. Then agents arrived, wired up to real tools through protocols like MCP, taking multiple steps toward a goal — and the instant an agent can pause, retry, or hand off to a human, it needs somewhere durable to keep its place. That's not optional anymore; it's the difference between a demo and a product. And past that, hovering as a direction rather than a shipped thing, is AGI and, further out, ASI — systems that would need memory in a sense closer to how we mean it: continuous, compounding, a lifetime rather than a session. We'll leave that frontier to the essay this course leads into. Here, we're building for the rung we're actually standing on.

Here's the uncomfortable foundation everything else in this course sits on: a language model has no memory of its own, ever. Every single call is stateless — you send text in, it computes a response, and it forgets you exist the moment that response finishes streaming. There is no persistent "it" between calls. Anything that looks like memory is something outside the model, engineered by whoever built the system around it.

Once you accept that, the interesting question becomes: what are the actual different jobs people are doing when they build that illusion? I find it clarifying to borrow a taxonomy from cognitive science — not because AI systems have minds, but because the four-way split maps cleanly onto four genuinely different engineering problems, and it's the same split the actual memory research (MemGPT's original paper, and the tools built on its ideas) reaches for too.

The most common way to build semantic memory is one you've likely already used for something else: retrieval-augmented generation, repurposed. If you've taken Embeddings in Depth or Semantic Search, you already know the mechanics — this chapter is just pointing the same machinery at a new target.

The idea, end to end:

Nothing here is exotic — it's the same embed-store-retrieve loop from semantic search, just with a different corpus (facts about one user or session, instead of a document set). What makes it "memory" rather than "search" is really just the write path: something has to decide what's worth remembering and turn a messy conversation into a clean, storable fact. That extraction step — turning "I guess I'd rather sit by the window, if that's not too much trouble" into prefers: window_seat — is where most of the real engineering effort goes, not in the vector database itself.

claude-mem is worth its own chapter because it's a good, concrete example of the pattern above, wired directly into a coding workflow rather than a chat product — and because "does it have memory" is exactly what people ask about their coding assistant after the fifth time they've re-explained a decision.

At a high level, tools in this family work the same general way, regardless of the exact implementation details (which move fast — always check the project's own docs for specifics):

The mechanism is exactly the RAG-as-memory loop from the last chapter, applied to your own working sessions rather than a customer's preferences: capture, summarise, embed, store, retrieve, inject. What makes it feel different in practice is where it sits — as hooks around the tool you're already using, rather than an API you have to call yourself — and that it's local-first: your session history isn't going to someone else's server by default.

Now the part I actually get asked about in design reviews: given all of the above, what do you reach for? Here's the landscape as it stands, compared on the questions that matter — not a feature checklist, but the actual trade-off you're making.

If there's one rule of thumb I'd write on a whiteboard, it's this: memory type should drive tool choice, not the other way round. Don't reach for Letta because it's the most sophisticated option; reach for it because your agent genuinely needs to decide for itself what to keep close. Don't reach for a LangGraph checkpointer for semantic memory; it isn't one, and it never will be — it solves a completely different problem (see The Serverless Memory Table).

Abstractions are easier to hold onto with a face on them. Here's the same four memory types, grounded in systems people are actually shipping, across a spread of industries — because the shape of the problem repeats far more than the domain vocabulary suggests.

Notice the pattern across all six: it's never "add memory," it's always a specific type of memory, chosen because of what breaks without it. A telecom support bot that only has working memory relitigates the same troubleshooting script every call. A healthcare scribe with only working memory forgets an allergy the instant the visit ends — which isn't an inconvenience, it's a hazard. The industry changes; the diagnostic question doesn't: what, specifically, is this system forgetting that it shouldn't, and which of the four types would fix that?

Theory earns its keep once you've built the smallest possible version of it. Here's a working semantic memory layer in well under 100 lines — the same embed-store-retrieve loop from chapter 3, made concrete. No framework, no dependencies beyond an embedding call and a list — small enough to actually understand every line of, and to extend once you do.

import numpy as np
from anthropic import Anthropic

client = Anthropic()
_store: list[dict] = []  # {"text": str, "embedding": list[float], "at": str}


def _embed(text: str) -> list[float]:
    # Swap in your real embedding model — this is illustrative, not a live call.
    # See "Embeddings in Depth" for the production version of this function.
    return client.embeddings.create(model="text-embedding-3-small", input=text).data[0].embedding


def remember(fact: str, at: str) -> None:
    """Write path: turn a fact into a retrievable memory."""
    _store.append({"text": fact, "embedding": _embed(fact), "at": at})


def recall(query: str, k: int = 3) -> list[str]:
    """Read path: return the k memories most relevant to the current query."""
    if not _store:
        return []
    q = np.array(_embed(query))
    scored = [
        (np.dot(q, m["embedding"]) / (np.linalg.norm(q) * np.linalg.norm(m["embedding"])), m)
        for m in _store
    ]
    scored.sort(key=lambda pair: pair[0], reverse=True)
    return [m["text"] for _, m in scored[:k]]


def respond(user_message: str) -> str:
    """Recall relevant memories, inject them, then answer — the whole loop."""
    memories = recall(user_message)
    context = "\n".join(f"- {m}" for m in memories) or "(nothing relevant remembered yet)"
    resp = client.messages.create(
        model="claude-opus-4-8",
        max_tokens=512,
        system=f"Relevant things you remember about this user:\n{context}",
        messages=[{"role": "user", "content": user_message}],
    )
    return resp.content[0].text

Zoom back out to the ladder from chapter one. Everything in this course — the taxonomy, claude-mem, the tool landscape, the code you just wrote — lives on the "Agents + MCP" rung. It's memory in service of a task: remember enough to act correctly, help a user, resume after a pause.

The open, genuinely unsolved research question sits one rung up. What would it take for a system to have something closer to a continuous self — memory that compounds across a lifetime rather than resetting per product, per session, per vendor? Nobody credible claims to have that solved. It's one of the real open problems standing between where we are and the systems people mean when they say AGI.

That's deliberately outside the scope of a hands-on course — it's the subject of an essay, not an architecture diagram. If you want to follow the thread from here to there — what general intelligence would actually require, what serious researchers disagree about, and what it might mean for the rest of us — that's where The Last Invention picks up.

You now have the full map: what memory actually is (four jobs, not one), how the tools people reach for day to day actually work under the hood, a clear-headed way to choose between them, what it looks like across real industries, and a working implementation small enough to hold in your head. The honest closing thought is the same one that opened this course — most of what reads as intelligence in a shipped AI system is memory, engineered well. Everything else is mostly the model doing what it always does.