Plain top-k over one embedding model is the floor, not the ceiling. Most real-world quality gains live right here, and they stack.

  query
    │  rewrite / expand   (fix vague or conversational queries)
    ▼
  ┌──────────────┬───────────────┐
  │ vector search│ keyword (BM25) │   ← hybrid: catch meaning AND exact terms
  └──────┬───────┴───────┬───────┘
         └──── merge ─────┘
                │  re-rank        (a cross-encoder reorders the top ~50)
                ▼
        metadata filter          (tenant, date, access)
                │
                ▼
        top-k passages  ──►  generation

Hybrid search — the biggest single win

Vectors match meaning but fumble exact strings — product codes, names, error IDs, acronyms. Ask for ERR_2041 and a pure embedding search may drift to "similar-sounding errors" and miss the exact one. Keyword search (BM25) nails exact terms but misses paraphrases. Run both and merge the results, and you catch what either alone would drop. For most corpora this is the largest quality jump you can make.

Re-ranking — cheap precision

The embedding search is fast but coarse. So retrieve wide — grab ~50 candidates cheaply — then run a cross-encoder re-ranker that reads each candidate against the query and reorders them, keeping the true top 5. It's a small addition with a large lift, because it judges relevance directly instead of by vector distance.

Query transformation — meet the messy query

Real queries are vague, conversational, or overloaded. A few high-value fixes:

• Rewrite a follow-up ("and the second one?") into a standalone question using the conversation.
• Expand acronyms and add synonyms so retrieval has more to match.
• HyDE — have the model write a hypothetical answer and search with that; a fake answer is often closer to the real passage than the terse question is.
• Sub-queries — split a multi-part question and retrieve for each part.

Tune k to your generator