Polp
← Back to glossary
RAG and search

Chunking

The ingestion step that divides a document into smaller passages so a retrieval system can index and return the most relevant context.

Business example

A 70-page employee handbook is split by headings into passages. A leave question retrieves the approved policy section instead of sending the entire handbook to the answer model.

How it works

  1. 1.Parse the source while preserving useful structure such as headings, pages, tables, and metadata.
  2. 2.Split the content into fixed, structural, semantic, or hierarchical passages.
  3. 3.Optionally overlap neighbouring passages so facts at a boundary keep enough context.
  4. 4.Embed and index each passage while retaining its source and permission metadata.
  5. 5.Test retrieval with representative questions and revise the strategy when passages are too broad or too fragmented.

Common misconceptions

Smaller chunks are always more accurate.
Small passages can improve precision but lose definitions, exceptions, or surrounding clauses needed to answer correctly.
One chunk size works for every document.
Policies, spreadsheets, manuals, slides, and conversations have different structures and should be evaluated with their real questions.

Related concepts

Retrieval-Augmented Generation (RAG)EmbeddingsVector databaseSemantic searchHybrid search

Related reading

RAG is not uploading PDFs: permissions, freshness, traceability, and reliable sources
How to evaluate internal AI search tools: a 15-question checklist

Sources

Reviewed:

Reviewed by: Javier Chulvi Bernad · LLM Engineer · Madrid