Prove It · Module 1: Why plausible answers can be wrong
A fluent answer is not evidence
An AI assistant generates an answer from the context available to it. That context can include your message, uploaded documents, search results and information returned by connected tools. Some answers rely mainly on patterns learned during training; others draw on retrieved material or calculations. The interface and task determine which tools are available.
Neither route guarantees accuracy. A model can produce a plausible detail that is wrong. A retrieved document can be outdated. A correct quotation can support a different claim from the one in the answer. A calculation can use the wrong input.
Your job is to connect the claim to evidence. Start with the statements that would change a decision: a deadline, price, policy requirement, quantity or person's responsibility. Check those against the original source. Do not use confidence, polished prose or the presence of a citation as a substitute.
A worked check
Fictional workplace example. Fernway's assistant says, “The supplier requires thirty days' notice, so cancel by Friday.” The attached contract says thirty working days and an amendment changes the renewal date. The assistant found a real clause but reached the wrong conclusion.
A useful follow-up is:
Show the exact clause supporting the notice period, its page and the document date. List any assumptions used to calculate the cancellation date. If an amendment conflicts, flag it instead of choosing silently.
Then open the contract and amendment yourself. The follow-up makes the answer easier to inspect; it does not make it verified.
At your desk
Choose three non-sensitive questions whose answers you can establish from reliable records: one date, one figure and one name. Ask the assistant, record its answer and check each claim. If it uses search or a file, inspect the cited passage. Notice whether its wording helps you distinguish what is sourced, inferred or unknown.
Repeat one question with a deliberately missing piece of context. Does it ask for clarification or quietly assume an answer? Record what happened rather than trying to prove that the tool always fails or always succeeds.
Write it down
- The claim I checked and why an error would matter:
- The original source, date and relevant passage:
- Whether the answer was supported, contradicted or unresolved:
- What I will check before using a similar answer at work:
Completion check: you can explain why an answer is supported without saying “the AI sounded sure”.
Put it to work: separate tone from evidence
Find an answer that sounds convincing. Underline the three claims that matter most, then check them independently. A confident introduction adds no support to those claims.