Test the bug, not the implementation

Learning outcome: Create a failing behavioural case before fixing a defect.

A useful regression test reproduces the behaviour that matters to a user. It should fail for the known defect and pass when the defect is fixed. A test that merely repeats the implementation's internal formula may preserve the same mistake.

Use independently worked expected values. Keep the test small enough that a failure is understandable. Where a defect spans storage or a network boundary, test the boundary rather than replacing every interesting component with a mock. After fixing the issue, run the focused case and the relevant surrounding checks. Broader tests should answer a concrete risk, not substitute for understanding the failure.

Worked example

A fictional app restores a saved stage value of 9 even though only stages 0, 1 and 2 exist. The page goes blank. A useful test supplies the invalid saved value and expects rejection or an explicit recovery state. Testing that the renderer accepts whatever number it receives would encode the defect.

Try it yourself

Define tests for missing stage, stages 0 and 2, and invalid values -1, 3 and a string. Describe the intended recovery without writing the implementation.

You can do this exercise on paper. If using an assistant, keep to the fictional material and an approved tool.

Check your understanding

Which test best protects against the defect?

1. The component uses the same variable name as before
2. The page has at least one div element
3. An invalid saved stage produces the specified recovery behaviour

Workbook

Download the course workbook to record your answer and compare it after the knowledge check.

Further reading

• Anthropic: effective agent design
• Anthropic: evaluating agents