The confirmation trap

The most dangerous question you can ask an AI tool is one that contains the answer you want. "Confirm that the deadline is the 31st." "Isn't it the case that this policy applies to contractors?" "This figure looks right, doesn't it?" The model is disposed to agree, and it will agree fluently, with supporting reasons it has just generated.

The fix is to ask open questions and to ask for disagreement. "What is the deadline?" not "Confirm the deadline is the 31st." "What are the strongest reasons this figure might be wrong?" "Argue against this conclusion." A model asked to find problems will find them, including real ones.

This applies to your own drafts most of all. If you ask "is this good," it will tell you it is good. If you ask "what would a sceptical reviewer challenge," you will get something useful.

In practice. A manager asked an AI tool whether her interpretation of a contract clause was correct. It said yes and gave three reasons. A colleague asked the same tool, in a fresh chat, what the strongest case against her interpretation was. It gave three reasons the other way, one of which was right. The clause did not mean what she thought.

At your desk. Take a conclusion you currently hold about something at work. Ask your AI tool, in a fresh chat, to argue against it as strongly as it can. Note whether anything in the argument is worth checking.

Write it down.

• The conclusion I tested:
• The strongest counter-argument it made:
• Whether it changed anything: