Hallucination and trust

Why models make things up

You already know the root cause from lesson one: a model predicts the most likely next token, not the most true one. Usually likely and true line up, since the internet it learned from is mostly people saying correct things, so it's right far more often than not. But when the likely-sounding answer and the true answer come apart, the model has no built-in preference for truth. It'll produce the plausible thing.

When it invents something false and presents it confidently, that's a hallucination. It's not lying; lying needs an intent to deceive, and there's no intent in there at all. It's a system doing exactly what it was built to do (produce plausible text) in a spot where plausible happens to be wrong.

Certain requests make it far more likely:

• Specifics it can't have. Exact statistics, precise dates, page numbers, phone numbers, legal citations, quotes. These are easy to fake convincingly and hard for the model to actually know.
• Things that don't exist. Ask it to summarise a report that was never written and it may cheerfully summarise a report that was never written, complete with invented findings.
• Gaps in its training. Anything niche, very recent, or private (your company's internal details) is a blank the model may fill with a confident guess.

The dangerous part isn't that it's sometimes wrong. It's that wrong and right arrive in the same confident voice. There's no tremor, no hedge, no tell. That's why you can't rely on how an answer feels.

Where citations help, and where they don't

The best defence built into modern tools is web search with citations. With search on, the tool fetches real pages and footnotes its claims with links. This helps: it grounds the answer in sources that existed, and it hands you the means to check.

But be clear-eyed about the limits, because a citation is not a guarantee:

• The link can be real but the claim mis-stated. The tool may cite a genuine page yet summarise it slightly wrong; say the source says "up to 20%" and the reply says "20%". The footnote looks authoritative; the number is off.
• The source itself can be wrong or out of date. A citation only proves somewhere said this, not that it's true. A linked blog post can be mistaken.
• Without search on, "sources" can be fabricated whole. A model answering from memory can produce a citation (title, author, even a URL) that simply doesn't exist. A link on the page is not proof it was ever fetched.

So citations turn "trust me" into "here's where to look", which is a real upgrade. But the checking is still your job. A footnote is an invitation to verify, not a substitute for it.

The verification habit: check the claim that matters, at its source

You can't and shouldn't fact-check every sentence; that would erase the time AI saves you. The workable habit is narrower and it's this: identify the one or two claims the decision actually rests on, and check each at its source.

Ask yourself: what in this answer, if it were wrong, would cause a real problem? Usually it's a small set: a figure going into a report, a date driving a deadline, a policy rule, a name, a legal or financial specific. Those are what you verify. The connective prose around them rarely needs it.

"At its source" is the important half. Verifying doesn't mean asking the model "are you sure?"; that just generates another confident paragraph, possibly repeating the same error. It means going to where the truth actually lives: the official page, the original document, the person who'd know, the spreadsheet the number came from. For anything grounded by web search, open the cited link and confirm the answer matches what the page really says.

A three-part rule of thumb for what always gets checked:

• Numbers: figures, percentages, prices, dates.
• Names and quotes: who said or did what, and exact wording.
• Anything with consequences: legal, financial, medical, safety, or "this goes to a customer/the board" claims.

"Wrong" is not the same as "outdated"

A useful distinction that changes how you react. Sometimes an answer is wrong: the model fabricated something that was never true. Sometimes it's merely outdated: it was true once, but the model's knowledge stops at its training cut-off and the world moved on. A price that was right last year, a policy since revised, a "current" figure that's a version behind.

The two need different fixes. Outdated is cured by grounding the tool in the present: turn web search on, or paste in the current document yourself. A wrong answer needs the source check above. Both share one cause worth remembering: the model can't tell you which mistake it's making, because it doesn't know it's making one.

A worked example: pressure-testing a confident answer

You ask a tool, with search off, for a fact to drop into a Fernway board paper.

What percentage of UK small businesses were using at least one AI tool in their operations last year? Give me the figure.

It replies with a crisp, specific percentage and a confident sentence. That specificity is the warning sign: it's exactly the kind of number that's easy to fabricate. So you ground it and demand something checkable:

Search the web for that figure. Give me the exact number, the organisation that published it, the year it refers to, and a direct link to the original source. If you can't find a reliable source, tell me plainly instead of estimating.

Now one of two useful things happens: it produces a real, dated source you can open and confirm, or it admits it can't find one, which tells you the first confident number was never trustworthy. Either way you're safe. The final step is always yours:

List just the single sentence from that source that supports the figure, so I can find it on the page and confirm it myself.

Try it now

Common mistakes

• Reading confidence as correctness. The whole trap in one line. The voice is identical whether the model is right or inventing. Never let a smooth tone stand in for a check.
• Trusting a citation without opening it. A link proves someone said it, not that it's true or that the tool quoted it accurately. Open the ones that matter.
• "Are you sure?" as verification. Interrogating the model is not checking; it can be confidently wrong twice. Go to the source.
• Checking everything, so checking nothing. If verifying feels impossible, you're aiming too wide. Verify the claims with consequences; let the prose be.
• Confusing outdated with wrong. A stale answer needs grounding in the present (search on, paste the current doc); a fabricated one needs a source check. Different fixes.

Keeping current

Tools keep improving at citing sources and admitting uncertainty, but none has eliminated hallucination, and none is expected to soon, so the habit stays essential. For how a specific tool handles accuracy and sourcing today, check its own documentation, for example Anthropic's notes on Claude's limitations. Correct as of 13 July 2026.