AI ToolsACADEMY
AI Tools Academy
0 / 117 (0%)

Retrieval engineering: beyond document chat

Finishing the course

Capstone · 30 min

Your final assignment

Design a retrieval test pack and diagnose whether failures occur before or after generation. Use the fictional examples from this course, or a non-sensitive scenario you are authorised to work on. Do not submit personal workplace records to complete the course.

Build your evidence pack

  1. Chunk boundaries and orphaned exceptions: Split the two-page rule into two different chunk designs. Write one question whose answer would fail under the weaker design. Include the source title and section in each chunk.
  2. Metadata filters and permission boundaries: Design metadata fields for three fictional policy documents. Include approval status, effective date and audience. Write a negative test proving a general user cannot retrieve the restricted document.
  3. Hybrid search for names, codes and meaning: Create four queries about two similar device codes, including one paraphrase of a symptom. Define the correct source for each before looking at search results.
  4. Reranking and retrieval recall: Build a ten-row evaluation sheet with expected source, retrieved yes/no, rank and answer supported yes/no. Fill three rows where the correct source is absent.
  5. Claim-level grounding and citation checks: Annotate the fictional answer claim by claim. Write a revised answer containing only supported information and clearly labelled unresolved questions.

Combine the five outputs in the downloadable course workbook. Write a short decision note explaining the problem, your chosen approach, the evidence used and one limitation that would matter in real use. Keep initial and revised versions where a check changed your answer.

Review against these criteria

  • Keep a rule and its qualifying context together.
  • Distinguish relevance filtering from access control.
  • Combine retrieval signals when exact matches matter.
  • Separate finding candidates from ordering them.
  • Check that each consequential claim is supported.

Ask a colleague or learning partner to challenge one assumption if available. If working alone, use the worked examples to review your reasoning and record the limits of that self-review. Completion is a personal learning record, not independent certification.

Change one condition

Choose one input or assumption from your original exercise and change it. Explain which part of the result must be recalculated, rechecked or withheld. Keep the revised answer and a short explanation of why the change matters. This is a transfer exercise: repeating the model answer without adapting it is insufficient.

Continue practising

Try a related Academy workspace or choose another course. The existing Practice Lab portfolio saves simulation work; keep this course workbook with your own learning records. Course completion progress remains in your browser and can be backed up through My Library.