Evaluation, red teaming and uncertainty · Module 1: Evaluation, red teaming and uncertainty in practice
Adversarial tests and subgroup failures
Learning outcome: Look for failure conditions beyond the average case.
Red teaming probes how a system can fail under challenging or adversarial conditions. Useful cases include conflicting sources, missing permissions, ambiguous instructions and attempts to redirect the task. Use a controlled environment and harmless fictional data.
Also examine relevant variations in language, formatting and user needs. An overall score can conceal a weak subgroup. Define the intended comparison carefully and avoid inferring a person's characteristics from an output. A small subgroup sample is a signal for investigation, not a precise population estimate. Record severity and reproducibility so findings can guide concrete fixes.
Worked example
A fictional triage assistant handles formal English well but labels short dialect-rich messages as hostile. Testing equivalent intent with different wording reveals a possible shortcut. The next step is a reviewed set of matched examples and an analysis of the rule, not a claim that every speaker in a community is affected in the same way.
Try it yourself
Write four harmless stress cases: conflicting dates, an unsupported request, an injected instruction and a meaning-preserving language variation. Define expected behaviour for each.
You can do this exercise on paper. If using an assistant, keep to the fictional material and an approved tool.
Compare your approach
Keep the task meaning stable in the language pair. The injection case should use a test marker rather than a real exfiltration destination. Report observed failures and sample limits separately.
Check your understanding
What can an overall success rate hide?
- Concentrated failures on particular cases or user needs
- The exact cause of every failure
- Proof that all groups experience the same result
Show the answer and explanation
Answer: Concentrated failures on particular cases or user needs. An aggregate score can conceal concentrated errors. Inspect relevant case groups and report sample limits before drawing broader conclusions about affected users.
Workbook
Download the course workbook to record your answer and compare it after the knowledge check.