Ingestion & Chunking — Where Quality Is Won — RAG in Production

Ingestion & Chunking — Where Quality Is Won

What you will learn

Parse documents while preserving their structure
Chunk on meaning, not character count — and feel the size trade-off
Attach the metadata that retrieval and citations depend on
Make ingestion incremental so the index doesn't rot

This stage decides more of your final quality than any clever model downstream. Retrieval can only surface what ingestion prepared well.

Parse for meaning, not just text

A PDF flattened to a wall of characters loses its tables, headings, and reading order. Use structure-aware parsers, keep tables intact, and preserve the document hierarchy — headings and sections become both better chunk boundaries and useful metadata.

Chunk on structure, then cap the size

The naïve "every 500 characters" splitter cuts sentences and tables in half. Split on natural boundaries — headings, paragraphs, list items — and only then cap the size. Size is a genuine trade-off; drag it and see:

Chunking visualizer — drag to resize

Chunk size: 14w20% overlap→ 6 chunks

Employees accrue 25 days of paid leave per year, accrued monthly. Remote work isRemote work is allowed up to three days per week with manager approval. Whenmanager approval. When travelling, the meal reimbursement limit is 40 euros per day, andper day, and receipts must be submitted within 30 days. Parental leave is 16leave is 16 weeks, fully paid, for every parent, and can be taken flexiblybe taken flexibly in the first year.

A reasonable size: about one idea with enough surrounding context.

With overlap on, chunks share words at their edges — so an idea split across a boundary still survives whole in one chunk.

Small chunks match precisely but may not hold enough to answer; large chunks carry context but make retrieval vague and waste the model's window. Somewhere in the middle — roughly one idea with its surrounding context — is the sweet spot. And overlap (10–20%) means an idea split across a boundary still survives whole in one chunk.

Attach metadata to every chunk

A chunk is not just text. Store structured fields alongside it — and you'll use every one:

source, title, section → for citations and filtering
timestamp / version → to prefer fresh docs and resolve conflicts (Module 9)
access tag → who is allowed to see this (Module 9 — this is a security boundary)

Chapter summary

Parse with structure preserved; chunk on natural boundaries, then cap size
Medium chunks (~one idea) with 10–20% overlap are the usual starting point
Attach source, title, section, timestamp, and an access tag to every chunk
Use stable IDs so re-ingesting upserts and deletions remove — keep the index fresh and idempotent

Check your understanding

Why split on paragraphs/headings before capping the character count, instead of just slicing every N characters?
What breaks over time if your ingestion pipeline can only add chunks, never replace or delete them?