How to prevent AI hallucinations in your work: the fact-check I run, and what the research says

A single factual error on your site and you're toast. That's why we have a fact-check agent. I strongly feel consensus in your content with evidence-based facts or citations is key. AI makes that harder, because a model states something false in exactly the same fluent tone as something true.

On the afternoon of 2 October, my fact-check agent flagged Saturn's moon count on this page, and I corrected it from 146 to 285. That evening I ran the corrected figure back through the method, and it was already stale: the count of record was 293.

The method is short: list the checkable claims, use AI to find what needs checking, then read the primary page and decide. In today's post I'm walking through how I run it, with that Saturn check end to end, and then what the 2025-26 research says reduces hallucinations, including where it pushes back on my method.

What an AI hallucination is, and why models make things up

An AI hallucination is a model stating something false as if it were true, and it isn't rare. Damien Charlotin keeps a database of court decisions in cases where AI produced hallucinated material, typically fake citations in filings, and on 2 October 2026 it held 2,125 of them. Those are decisions, not every fake citation ever filed, so the real number is higher. Air Canada's chatbot gave a customer incorrect information about its bereavement-fare policy, and Air Canada argued it couldn't be held liable for what its chatbot said. The British Columbia Civil Resolution Tribunal found against it on 14 February 2024 and awarded about CAD 650 in damages.

It comes down to how the models work. A large language model predicts the next word, or strictly the next token, from the words that came before it. There's no table of facts it looks up. When it gets something right, the pattern was well rehearsed in its training data. When it gets something wrong, the sentence comes out every bit as fluent.

In a paper OpenAI published in September 2025, Adam Kalai and colleagues argued that models hallucinate because "the training and evaluation procedures reward guessing over acknowledging uncertainty". OpenAI's blog post shows it on SimpleQA, a set of short questions with one right answer, where gpt-5-thinking-mini declined to answer about half the time and the older o4-mini almost never did. The older model looks marginally more accurate, at 24% against 22%, and gives about three times as many wrong answers.

Not every wrong answer is wrong in the same way.

A factual hallucination is the wrong number, name, date or claim. Attribution is a source that doesn't exist, or a real one that doesn't say what the model says it does. Capability is the model telling you it can do something it can't, such as read your CRM. Agreement is the model accepting a flawed premise in your question and answering inside it.

Attribution turns up even in professional tools. Stanford's RegLab and HAI researchers call it a "misgrounded" answer: a real citation that doesn't support the claim. They found it in legal research tools sold to lawyers, which is why a citation proves nothing until you've read what it points to.

How to prevent AI hallucinations: the check I run

The check I run came out of a mistake in June. On my sim racing site, a feature claim an LLM had made up became the thesis of an article. It shipped, and the tool's developer publicly called it "AI non-sense". After that I had 316 articles on the site checked, and 66 needed correcting. Since then I've worked to one rule: the LLM is a detector, not a source. A grounded answer (the model searched the web before answering) is a hypothesis, a claim that carries weight gets confirmed by a second method, and if two passes disagree, the claim is unverified.

On 2 October I ran the same check over every live article on houtini.com: 64 articles and 1,617 claims, with 49 articles corrected live the same day. The method has six steps, shown below, and the sweep used all of them except the second model, which came in for the Saturn check.

List the claims that carry weight

A paragraph can't be checked but a claim can, so the first job is to list the claims. My agents pull the live page and write down every checkable one on it: versions and release dates, model IDs and context windows, prices, environment variable names, install commands, package names, hardware numbers, "X supports Y" claims and URLs.

Each agent takes one article and does QA only. It lists and checks; it doesn't edit anything and it doesn't publish. On 2 October that meant 64 agents, run in 13 batches of five, and 155 fixes came back graded by severity: 64 high (wrong), 72 medium (stale) and 19 low (cosmetic).

Check your own tools against the source, not a model

Some claims shouldn't go anywhere near a model. My own tools are checked against the local repository - the README, the CHANGELOG and package.json - and never asked of an LLM. If the version number is in package.json, that's the answer.

The same goes for your own business. Your prices and policies already sit in a pricing page or a policy document you control, and that's what you check the claim against, because a model only knows what it has read about you.

Use a grounded model as a detector, not a source

For everything else I use a grounded model as the detector, through Gemini MCP. An MCP server is a plug-in that gives Claude a tool, and this one lets Claude call Gemini. In the README's words: "Grounding is on by default. Gemini searches Google before it answers, so you get this month's information rather than a training-cutoff guess, and the sources come back as markdown links."

My agents call Gemini 3.1 Pro (gemini-3.1-pro-preview) through gemini_chat at temperature 0.2, with three to five claims per call, and ask for the primary source URL behind each one. Grounding redirect links, the vertexaisearch.cloud.google.com ones, don't count as sources. The brief my agents work from puts it in capitals: "Gemini is a DETECTOR, not a source."

An article of mine quoted a "9.52/10 user rating" for a tool, and when the agent asked where the figure came from, Gemini's grounded search cited that same Houtini article as the origin. The vendor's own homepage settled it - "9.52/10 User Rating" is there - so the claim stayed, attributed to the vendor.

Get a second opinion from a separate model

The second opinion comes from a different model altogether. houtini-lm is my MCP server for handing bounded work from Claude to another model. Mine runs through a LiteLLM router to DeepSeek V4, OpenAI's models and the local models on my own machine, which has two modded 48GB RTX 4090s. Its README puts the case: "A different model reads your code with different blind spots, and it costs you next to nothing to ask." That line is about code review, and it holds for a claim too.

I don't ask the second model to rewrite anything. I ask for a verdict of correct, incorrect or cannot verify, what it believes the figure is and as of when, its confidence, and the primary source that would settle it. In the Saturn check below, that last answer was the useful one.

Agreement from a second model proves less than you'd think, and I'll come to the research on why further down.

Read the primary page, then decide

Nothing changes on the page until a primary source says so. A primary source is the organisation that keeps the fact: the vendor's docs, the court's order, the body that keeps the count. A news story about it isn't one. The rule my agents work to is that "a correction needs a named, linkable PRIMARY source (vendor docs, repo, pricing page, official post) confirmed by a second method." The second method is a Brave search, then Firecrawl to fetch the primary page.

Then decide whether to keep it, correct it or flag it. If you're uncertain, flag it and leave the text alone; the brief's rule is "Never invent a correction."

The sweep found two things AI had put on my own pages that no source backed. One was a quote attributed to Anthropic that had been a paraphrase in research notes and turned into a quote along the way. The other was a block of terminal output presented as what a tool prints, which it doesn't. Both were removed or replaced with the sourced version.

What the check caught on the RAG piece in May

Before my piece on RAG went live in May 2026, a check of its named claims caught three errors the drafting model had written. RAG is retrieval-augmented generation: the model is handed documents to answer from.

The draft said Claude Sonnet 4.5 supports a 1M-token context window. It's 200k; the 1M model was Sonnet 4.6. It put OpenAI's GPT-5 family at 1M tokens, where OpenAI's GPT-5 model page says 400k. And it described Anthropic's Contextual Retrieval as prepending a document-level summary to every chunk. Anthropic's engineering post from September 2024 describes something different: chunk-specific context that places each chunk within its document.

I re-checked the first one against Anthropic's docs on 2 October, and it still holds: "Other Claude models, including Claude Sonnet 4.5, have a 200k-token context window." The same docs now say Sonnet 4.6 and the Claude 5 line default to 1M, with no beta header, and OpenAI has since moved on to the GPT-6 family.

A real check: how many moons does Saturn have?

The sentence I checked was "Saturn has 285 known moons." The 285 had gone in on the afternoon of 2 October, taken from the IAU's March 2026 announcement.

That evening I checked it one tool at a time, and I asked the second model before the grounded detector. I used houtini-lm to pass the sentence to DeepSeek V4 Flash, with no web access and the temperature at 0.1, and asked it for a verdict, what it believed and as of when, its confidence and the source that would settle it.

DeepSeek said incorrect. It believed the count was 274, as of the March 2025 announcement, and gave its confidence as medium, because moon counts "are revised frequently" and it couldn't rule out a later batch of discoveries after its knowledge cutoff. Then it named the primary that would settle it: the IAU Minor Planet Center catalogue, cross-checked against NASA JPL Solar System Dynamics.

So it was wrong on the number and right that it couldn't know, and it named the sources I'd need. Next came the grounded detector. I asked Gemini the same question through gemini_chat, with grounding on, in a shorter version of the format.

Gemini said correct: "285, as of March 2026", pointing at minorplanetcenter.net. Earlier the same evening, in a different grounded call, the same model had said Saturn "now has 293 confirmed moons", citing NASA's Saturn page.

A plain Brave search didn't settle it either. The top snippets for "how many moons does Saturn have" said 274 (NASA's own page), "53 confirmed and named moons, though as many as 150 may exist" (Universe Today), 274 "as of March 11, 2025" (Caltech's Cool Cosmos) and "over 250" (ESA). Two of them agreed on 274, and neither was current.

At that point I had 274, 285 and 293 in play, so I fetched the primary pages with Firecrawl.

IAU, 26 March 2026 - iau.org/IAU/IAU/News/Ann2026/MPC-New-Moons-Saturn-Jupiter.aspx
  "...taking the total known Saturnian moons to 285."

NASA JPL Solar System Dynamics - ssd.jpl.nasa.gov/sats/discovery.html
  Satellites of Saturn: 293

NASA Science, Saturn Moons (modified 22 Sep 2026) - science.nasa.gov/saturn/moons/
  Page body:        "Saturn has 293 confirmed moons in its orbit as of August 2026, ..."
  Meta description: "Saturn has 274 confirmed moons in its orbit, ..."

The IAU's announcement of 26 March 2026 confirmed 11 more moons. JPL's Planetary Satellite Discovery Circumstances table lists only satellites "officially recognized by the International Astronomical Union", and NASA's Saturn Moons page, modified on 22 September 2026, links its 293 straight to that table.

When the agent cross-checked the 285 against NASA's page that afternoon, it read the page's meta description (the text a search engine shows as the snippet). That still said 274, while the body said 293. 285 was right in March and stale by August.

Is a second model a real second opinion?

Not on its own.

Elliot Kim, Avi Garg, Kenny Peng and Nikhil Garg looked at errors across more than 350 LLMs (ICML 2025). On one leaderboard dataset, when two models both got a question wrong, they gave the same wrong answer 60% of the time. They also found that "larger and more accurate models have highly correlated errors, even with distinct architectures and providers." Shashwat Goel and colleagues (2025) found the judging side has the same problem: when one model scores another, it favours models similar to itself.

Asking one model the same question several times isn't the way out either. Sebastian Farquhar and colleagues at Oxford showed in Nature in June 2024 that semantic entropy, which measures how much the meaning of a model's answers varies across repeated samples, catches confabulations with an AUROC of 0.79. On that scale 0.5 is chance and 1.0 is perfect. But in their words it "does not help when LLM outputs are systematically bad", so a model that's wrong the same way every time passes.

What my run adds

So two models agreeing from memory isn't consensus between independent sources. My run qualifies that: the second model earned its place by disagreeing and naming the source that would settle it, and I still had to go and read that source. None of these studies tests a second model used alongside a fetched primary page, which is how I use houtini-lm.

What the research says reduces AI hallucinations

Ranked by strength of evidence, the 2025-26 research puts grounding a model in a source first: it's the best-measured fix, and it still leaves a remainder. Letting a model say "I don't know" comes next. Sampling a model several times and comparing the answers catches some errors and misses the consistent ones. Checking each claim against a quote or citation has vendor numbers behind it but no independent measurement. Human review against primary sources has strong evidence of the damage done when it's skipped, though none of these studies measures how much it cuts errors. Cross-checking with a second model is the weakest on its own, and upgrading to a newer model shows no consistent reduction at all.

Where the evidence agrees

Grounding reduces hallucination and doesn't remove it. Stanford's RegLab and HAI team (Varun Magesh, Daniel Ho and colleagues, in the Journal of Empirical Legal Studies, June 2025) found the AI research tools made by LexisNexis and Thomson Reuters "each hallucinate more than 17% of the time", against 58% to 82% for general chatbots in an earlier study. Vectara's leaderboard measures how often a model adds unsupported content when summarising a document it has been handed, and still finds rates from 1.8% to 24.2% across around a hundred models; Vectara scores it with its own model. Google DeepMind's FACTS leaderboard had a best average of 75.7% on 28 September 2026, and Google built that benchmark. The suite has four benchmarks, and only one of them lets the model search the web. Even the best system gets roughly one item in four wrong.

A real citation isn't a verified claim, and Anthropic's own documentation tackles it from the other side: have the model back each claim with a quote, and "If it can't find a quote, it must retract the claim."

Letting the model say "I don't know" lowers errors. OpenAI offers its SimpleQA numbers above as evidence, and Anthropic's docs say: "Explicitly give Claude permission to admit uncertainty."

People still have to check, and courts sanction the ones who don't. In Withers v. City of Aberdeen, a sanctions order of 8 June 2026 in the Northern District of Mississippi listed six invented citations across three filings from both sides. All four lawyers were sanctioned, including the two local counsel, who hadn't used AI at all and had signed the filings without checking the citations.

Where the experts disagree

Can hallucination be eliminated? OpenAI's blog post says accuracy "will never reach 100%", because some real-world questions can't be answered, but that hallucinations aren't inevitable, because a model can abstain when it's uncertain. Amin Karbasi and colleagues (2025) argue from theory that automated detection is "fundamentally impossible" for most cases if the detector is trained only on correct examples. By their account it needs expert-labelled examples of errors. Vectara, which runs one of the leaderboards, says it directly: "We do not claim to have solved the problem of hallucination detection".

Does RAG fix it? The legal vendors described their tools in terms like "hallucination-free". Stanford measured more than 17%.

Do newer models hallucinate less? Sometimes. OpenAI's own system card for o3 (April 2025) shows it hallucinating on 33% of PersonQA questions, which are questions about people, against 16% for the older o1. It made more claims overall, right and wrong. Google says Gemini 3 Pro cut errors on the FACTS Search benchmark by 55% compared with Gemini 2.5 Pro.

My routine checks every claim, whichever model wrote it, because a newer model isn't reliably a more accurate one. The three errors in May came from the drafting model of the day. On 2 October, the stale Saturn figures came from search snippets, a grounded answer and the second model's memory.

Where fact-checking AI still falls down

Nothing I've found measures what a single factual error does to a website. The hard evidence of consequences is in court.

Shared misconceptions

When the public web is wrong, the grounded sources are wrong together. Consensus isn't truth, and a detector that searches the web will agree with the web. It's the same blind spot Farquhar's team describe, where a model is consistently wrong because it learned something wrong. That's why my consensus has to include the primary page, and high-stakes work needs a person with subject authority on top.

Recency

A claim about something that happened last week may sit in the gap between what has happened and what the sources say. NASA's Saturn page showed it on 2 October: the body said 293, while the snippet that search engines show still said 274. Go to the primary - the announcement, the filing, the release - and read it.

Anchoring

A question phrased with the conclusion in it tends to get that conclusion back, confidently. Phrase it neutrally. Both prompts below ask for "correct / incorrect / cannot verify", which gives the model an answer other than agreeing with you.

Copy the prompts I use

Both prompts I used for the Saturn check are below, with the claim swapped for a placeholder. Run the second-opinion prompt through houtini-lm, or any second model you have to hand. Run the detector prompt through Gemini MCP with grounding on. Then open the URL it gives you and read the page yourself.

# 1. Second opinion - houtini-lm, or any second model (no web access needed)
# System: You are a fact-checker reviewing a draft article before it is published.

Check this sentence from a draft article: "<paste the sentence>"

Answer in exactly this format, no preamble:
Verdict: correct / incorrect / cannot verify
What I believe the figure is, and as of when:
Confidence (low / medium / high):
The primary source that would settle it:

# 2. Detector - Gemini MCP, gemini_chat with grounding on
# (in the Saturn run the last line said "keeps the count")

Check this sentence from a draft article: "<paste the sentence>"

Answer in exactly this format, no preamble:
Verdict: correct / incorrect / cannot verify
The current figure, and as of when:
The primary source URL (the organisation that keeps the fact, not a news story):

If you run a team that publishes or sends AI-assisted work, there are two questions worth asking of every workflow. Did anyone read this output before it shipped? And were the claims that carry weight checked against their primary sources? If you'd like those answered across a whole company, that's what the Houtini AI Audit is for: a structured pass through the workflows that already use LLM output - what's checked, what isn't, and where the check is missing. If you'd rather ground the model's answers in your own documents to begin with, my piece on RAG is the place to start.