Chunk boundaries and orphaned exceptions

Learning outcome: Keep a rule and its qualifying context together.

Retrieval-augmented generation, or RAG, supplies retrieved material to a model before it answers. Many systems divide documents into chunks for indexing. A chunk is a retrieval unit, not necessarily a complete argument. Splitting between a rule and its exception can make the rule appear more general than it is.

Inspect the extracted text before tuning the model. Preserve section headings, document identity and links to adjacent passages. Try chunking strategies on questions that need exceptions or table headers. Larger chunks retain more context but may include irrelevant material. Smaller chunks can be precise but incomplete. Choose using representative tests rather than a universal character count.

Worked example

A fictional handbook says “Travel meals are reimbursable” at the bottom of one page. The next page says “Only on overnight assignments and with receipts.” A retrieved first-page chunk supports a misleading general answer. Keeping the condition attached, or retrieving the neighbouring section, changes what the evidence actually permits.

Try it yourself

Split the two-page rule into two different chunk designs. Write one question whose answer would fail under the weaker design. Include the source title and section in each chunk.

You can do this exercise on paper. If using an assistant, keep to the fictional material and an approved tool.

Check your understanding

What is the risk of retrieving a rule without its exception?

1. The answer may wrongly apply the rule to excluded cases
2. The response will always be too short
3. The model will automatically retrieve the missing page

Workbook

Download the course workbook to record your answer and compare it after the knowledge check.

Further reading

• Elastic: hybrid search
• Elastic: semantic reranking