Evaluation, red teaming and uncertainty · Module 1: Evaluation, red teaming and uncertainty in practice
Golden sets and test contamination
Learning outcome: Keep evaluation cases independent enough to remain informative.
A golden set is a curated collection of inputs with expected outcomes or scoring criteria. It should represent the intended task, including difficult and unsupported cases. A reference answer is only as reliable as the process used to establish it.
Separate development cases from held-out evaluation where possible. If every prompt revision is tuned against the same test set, performance may reflect familiarity with those cases. Record provenance and changes to the expected answers. Include newly discovered failures without pretending the set is a complete model of the real world. A small set is useful for diagnosis, but weak grounds for sweeping claims.
Worked example
A fictional assistant passes twenty questions used repeatedly during prompt development. Five new questions contain missing source information, and it invents answers to all five. The original score measured a narrower capability than the team claimed. The held-out cases expose the need for an explicit insufficient-evidence response.
Try it yourself
Draft a ten-case golden set outline with normal, ambiguous and unsupported questions. Reserve some cases from prompt tuning and write how their answers will be established.
You can do this exercise on paper. If using an assistant, keep to the fictional material and an approved tool.
Compare your approach
Use approved source records or independently reviewed expected behaviour. Unsupported cases should expect an appropriate limitation, not a fabricated complete answer. Record which cases were used during development.
Check your understanding
Why reserve held-out cases?
- To reduce the chance that improvement only fits repeatedly tuned examples
- To avoid documenting expected answers
- To guarantee the system will generalise perfectly
Show the answer and explanation
Answer: To reduce the chance that improvement only fits repeatedly tuned examples. Held-out cases offer a check beyond repeatedly tuned examples. They still need valid reference answers and representative coverage; separation alone does not guarantee generalisation.
Workbook
Download the course workbook to record your answer and compare it after the knowledge check.