Embeddings & the Vector Store
What you will learn
- Understand what an embedding is and why similar meanings sit close together
- Choose an embedding model with the trade-offs that matter
- Know what a production vector store must provide
- Avoid the most expensive mistake in RAG: the re-embedding trap
Meaning as geometry
An embedding converts text into a vector — a long list of numbers — arranged so that "vacation policy" and "how much leave do I get" land near each other, even with no shared words. That's the magic you saw in the Module 1 playground, made concrete. Retrieval is then just: embed the query, find the nearest document vectors.
Choosing an embedding model
This is a trade-off across a few axes:
- Quality — how well it captures meaning on your kind of text (legal, code, and casual prose differ).
- Dimension — bigger vectors can be richer but cost more to store and search.
- Context length — the longest chunk it can embed at once.
- Hosted vs local — an API call per embedding, or a model you run yourself.
The vector store
The store is where chunks live and get searched. Managed (Pinecone, Weaviate), self-hosted (Qdrant, Milvus), or bolted onto your existing database (pgvector). The right pick depends on scale and how much you want to operate — but every serious store must provide the same three things:
- Approximate nearest-neighbour (ANN) index — usually HNSW — for sub-second search across millions of vectors (checking every vector exactly would be too slow).
- Metadata filtering at query time — by source, date, tenant, access level.
- Upsert and delete — so the freshness work from Module 4 actually lands.
Chapter summary
- Embeddings place text in space so similar meanings are neighbours; retrieval finds the nearest
- Choose an embedding model by benchmarking on your own data — the choice is sticky
- A production store needs ANN search, metadata filtering, and upsert/delete
- Query and document embeddings must use the same model version; plan for costly re-embeds
Check your understanding
- Why can an embedding match "how much leave do I get" to a document about "vacation policy" that shares no words?
- Why is switching embedding models after indexing a million documents so costly?