AI research reports: what happens when AI checks AI
A product marketing lead built an agent system to test product ideas, then found the AI errors were getting subtler. Six typical mistakes show how.

Fake sources are the easy problem. A product marketing lead who built a multi-agent tool for testing ideas says the harder ones are quotes that are real and conclusions that are nonsense.
The background is a string of embarrassing consulting cases. Deloitte handed the Australian government a report with nonexistent sources, agreed to return its final payment, and the ministry said the recommendations hadn't changed. Similar problems have turned up in reports from Deloitte in Canada, EY and KPMG. The author's point: AI made generating text cheaper, but not checking it.
His system is called "Hypothesis Polygon" (Полигон гипотез). You bring an idea and context. Agents break it into claims that can be checked, look for support and contradictions, and suggest what to drop and what to rework. It can also gather extra material and run synthetic experiments.
What kinds of AI mistakes did he find?
He lists six. Two out of ten customers unhappy with an integration became "most customers". Four people saying a mobile app would be "nice" became confirmed demand, partly because the team's brief already said they wanted one. Three agencies backing a 25% market growth figure all repeated one manufacturer press release. A report on why customers don't buy answered a question about renewals. Two English-language searches turned into "no competitors", while a local company sells the feature. And a bank's 15% churn drop was an invented detail with no support in the materials.
How does he catch them?
Generalisations now need a list of who said what, and a script recounts the participants. The brief's own assumptions get checked, and each main conclusion needs the strongest argument against it. A source registry counts independent confirmations. Searches must try to disprove a claim and log where they looked. For factual claims, a script finds the quote in the cited text and recalculates figures, then a separate agent checks the quote supports the claim.
There is a cost. Tighter rules made the system more cautious and less useful, and the author says the checks themselves sometimes err. Early on, he just added instructions like "don't generalise" and asked for a reread. In one analysis, "three out of ten" stayed three even though it was wrong.
Apps worth a look
Tap the right tiles, add them up, hit the target. Easy to learn, hard to master. Free, for all ages.
Talk to an AI conversation partner; every sentence comes back corrected and explained in your language. 9 languages, A1 to C2.
Open EDIFACT and X12 messages, see every segment explained, convert to JSON/XML, export PDF and Excel. Runs locally, no upload.
Pick who it is for, the occasion and their personality. The search engine reads product descriptions from local shops and ranks what fits. Free, no sign-up.
More from this day
See all →
Keisei Skyliner Data Breach: Up to 3.37 Million Customers Hit

Blade Runner AAA game: rights holder Alcon is in talks

Gentle Monster smart glasses with Google sell out before launch

Revolut Italy placed under court administration for a year
More in: AI
See all →
Alibaba Qwen 4 Max tops Chinese AI agent rankings, but is closed

ChatGPT Wallet lets OpenAI's dots agent pay without your card number

Russian AI detectors Dumate and Antiplagiat fooled by typography
