Evaluation, red teaming and uncertainty · Module 1: Evaluation, red teaming and uncertainty in practice
Uncertainty, abstention and release decisions
Learning outcome: Let a system stop when the evidence is insufficient.
Abstention means declining to make a prediction or answer under defined conditions. It can be a useful outcome when sources are missing, conflicting or outside the system's scope. Measure whether abstention happens for the right reasons, not merely how often the system answers.
A confidence score needs interpretation and validation; a model saying “95% sure” is not automatically a calibrated probability. Define release thresholds by task and failure severity. Track coverage alongside correctness so a system cannot appear excellent simply by refusing every difficult case. Keep a human escalation route and a record of unresolved risks.
Worked example
A fictional assistant answers 60 of 100 questions, with 57 supported answers. Its answered-case accuracy is 95%, but coverage is 60%. If the forty unanswered cases include routine supported questions, the system may be too cautious. If they are genuinely unsupported, abstention may be appropriate. The score alone cannot decide release.
Try it yourself
Write a release note reporting both 57/60 supported answers and 60/100 coverage. Name one investigation needed before deciding whether to expand access.
You can do this exercise on paper. If using an assistant, keep to the fictional material and an approved tool.
Compare your approach
Report 95% among answered cases and 60% coverage. Inspect the unanswered cases and the severity of the three unsupported answers. Avoid presenting 95% as success across all one hundred questions.
Check your understanding
What should be reported alongside accuracy among answered cases?
- The length of the system prompt
- Coverage and the reasons for abstention
- Only the most confident answer
Show the answer and explanation
Answer: Coverage and the reasons for abstention. Accuracy among answered cases excludes abstentions. Coverage and abstention reasons reveal whether the system is usefully cautious or simply avoiding cases it should handle.
Workbook
Download the course workbook to record your answer and compare it after the knowledge check.