Why one AI agent will confidently lie to you, and how a small team of them catches it
Research from MIT and Google and a live agent system point the same way. The fix for AI that invents things isn't a better prompt. It's several agents, each with its own brief, checking each other's work.
Your AI tool will sometimes invent a price, a policy or a fact, and state it with complete confidence, because one model can't see its own mistakes. Researchers at MIT and Google tested a fix: in their 2024 ICML paper, Du and colleagues had several AI models answer independently and then debate each other's answers, which improved reasoning and reduced invented facts compared with a single model.
I learned what that costs the hard way. AI agents "hallucinate", the polite word for inventing something that sounds completely true. I gave one of my agents a real job on its own. It sounded confident and the output looked clean. It was also wrong in places I didn't catch, because nothing had looked at its work twice.
What changed when I added more agents
Then I put several agents on the same job, each with its own context and its own brief, and the errors started getting caught.
They check each other. When one invents a fact, another is often holding the real source and flags it. When one guesses, the others can push back until the version that matches the evidence wins.
That's not a trick. It's the same logic behind Leo Breiman's random forest in machine learning, where no single decision tree decides anything and the forest votes. The same idea underpins forecasting markets and juries. Several independent judgments, each allowed to be wrong, are more likely to reach the right answer than one confident voice, as long as each is right more often than not. Mathematicians call this the Condorcet jury theorem.
The important word is independent. Agreement only means something when each agent looked on its own instead of copying the others.
What it looks like in practice
My social replies now go through a small team of agents instead of one model:
- One agent finds conversations worth joining and drafts a reply.
- A separate agent checks every factual claim against a catalogue of things I've actually done, with sources.
- Another grades structure, length and tone, without seeing how the draft was produced.
- A last one checks the reply against thousands of earlier ones, so I never say the same thing twice to the same person.
The first draft still comes back with confident mistakes. Only 10–20% of the posts the system finds survive its relevance and safety screen, and a batch ships only when all four reviewers agree. I still approve what gets published. That system has published 15,621 replies this way. I wrote it up, including what broke, in the case study.
Why this matters more for small businesses
For a small business, the tempting route is one AI tool that does everything. It's fast and it sounds impressive. But, as the debate study suggests, a single model struggles to see its own blind spots, in the same way one person can't proofread their own sales copy. They read what they meant to write, not what's on the page.
The fix the research points to is not a cleverer prompt. It's a second and third independent pass, each with its own context and its own seat at the table.
You can apply this without building anything elaborate:
- Separate writing from checking. The prompt or tool that drafts a customer email should never be the one that approves it.
- Give the checker the source, not the draft's reasoning. A reviewer that sees the original price list, policy or contract will catch invented details. A reviewer that only sees the draft will be persuaded by it.
- Check for one kind of error at a time. "Are the facts right?" and "Is the tone right?" are different jobs. Asking one pass to do both means it does neither well.
- Keep a human on the final step for anything a customer will see, at least until the error rate is low enough that you'd trust a junior employee with it.
None of this has to be expensive. In this setup, a cheaper, faster model handles the simpler jobs and the stronger one is saved for the final check, and the whole team runs on one subscription of roughly US$100 to US$130 a month. A second pass usually costs far less than one wrong price reaching a customer.
This is the last step of the method I use on every project, Audit, Optimize, Automate, Verify: the checking never stops.
My standard for AI output isn't one smarter answer. It's several honest answers, allowed to disagree, until the right one survives.
Originally posted on LinkedIn
4,865 impressions and 27 comments. The discussion in the comments is worth reading too.