Semantic Search & Embeddings

How Dense Embeddings Represent Meaning

From word vectors to sentence transformers — what embeddings actually are, how cosine similarity works, and why they enable semantic search.

Choosing and Comparing Embedding Models

BGE, E5, OpenAI, Cohere — how they differ, how to benchmark them for your use case, and what to look for in MTEB scores.

Vector Database Fundamentals — HNSW and IVF

How approximate nearest neighbour algorithms work under the hood, and what trade-offs to expect from HNSW vs IVF in production.

pgvector — Vector Search Inside PostgreSQL

Add a vector column to PostgreSQL, create an HNSW index, run similarity queries, and filter by metadata — all in SQL.

Dedicated Vector Stores — Qdrant and Pinecone

Upsert, query, and filter vectors with payload metadata in Qdrant (local Docker) and Pinecone (managed cloud).

Hybrid Search — Dense + Sparse + Re-Ranking

Why pure semantic search fails for exact terms, how BM25 fills the gap, how Reciprocal Rank Fusion combines rankings, and how a cross-encoder re-ranker makes the final list trustworthy.

Building a RAG Pipeline End to End

Chunk, embed, index, retrieve, and generate — a complete working RAG pipeline from a document folder to grounded LLM answers, with citations.

Production RAG — Evaluation, Caching, and Drift

Measure RAG quality with RAGAS, protect against embedding model updates with drift detection, add caching to cut costs, and know what breaks in production.