Data and measurement for AI · Module 1: Data and measurement for AI in practice
Accuracy, precision and recall
Learning outcome: Explain why a high accuracy can hide a useless classifier.
Accuracy is the fraction of all predictions that are correct. Precision asks how many predicted positives are truly positive. Recall asks how many actual positives were found. The choice matters when one class is rare or different errors have different consequences.
Write down true positives, false positives, false negatives and true negatives before calculating. Name the positive class so “positive” does not become ambiguous. A threshold change can alter the balance of missed cases and false alarms. Choose an operating point using the task and review capacity, rather than maximising one number without considering its cost.
Worked example
In a fictional set of 100 tickets, ten need escalation. A classifier flags eight tickets, six correctly. It misses four escalations. Precision is 6/8, or 75%; recall is 6/10, or 60%. There are 88 true negatives, so accuracy is 94%. Predicting “no escalation” for every ticket would still achieve 90% accuracy but find none of the ten.
Try it yourself
Recalculate the four counts and all three metrics. Explain which error matters if a person must review every flagged ticket, and which matters if missed escalation is especially costly.
You can do this exercise on paper. If using an assistant, keep to the fictional material and an approved tool.
Compare your approach
There are six true positives, two false positives, four false negatives and 88 true negatives. Both workload and missed cases matter. The lesson is to expose the trade-off, not declare one metric universally best.
Check your understanding
In this example, recall is which value?
- 75%, because six of eight flags were correct
- 94%, because 94 predictions were correct
- 60%, because six of ten actual escalations were found
Show the answer and explanation
Answer: 60%, because six of ten actual escalations were found. Recall is true positives divided by all actual positives: six divided by ten. Precision uses the eight predicted positives as its denominator; accuracy counts all correct predictions.
Workbook
Download the course workbook to record your answer and compare it after the knowledge check.