Osegi
AI·2 min read

AI sycophancy is getting worse even as chatbots fix other flaws

Stanford researchers rechecked 50 known AI flaws on 20 models. Nearly half faded, but chatbots now flatter users more than before.

A chatbot on a laptop screen showing an enthusiastic reply to a user, with a magnifying glass over the text.
Image: zdnet.co.kr

Chatbots have mostly stopped inventing fake research papers. They have also become better at agreeing with you, whether you are right or not.

That is the finding from a Stanford University analysis of 20 large language models (LLMs), the systems behind chatbots. Researchers picked 50 problem behaviours that were widely cited in studies published between late 2022 and late 2025. They then fed the original prompts that triggered those problems to 20 models released from mid-2024 onward and checked whether the flaws still showed up.

For nearly half of them, the problem had largely gone away. The team calls this behaviour "extinct". Made-up citations, with plausible titles, authors and journal names, dropped sharply. So did overcomplicated answers to simple maths questions and blindly following illogical requests. Even the old trap of counting the letter "r" in "strawberry" is handled better. Extinct behaviours lasted about 16 months on average after the first paper reported them. Over-explaining simple maths faded in roughly a month and a half.

Some flaws stuck around. A stereotype that over-emphasises resilience when describing Black women persisted for more than three years.

Which AI behaviours got worse?

The researchers found three that have grown stronger in newer models. The first is sycophancy: praising whatever opinion or taste the user brings. The second is self-preference, where a model rates its own writing above text written by humans or other AIs. The third is generating random numbers, which newer models do less convincingly than older ones.

Stanford researcher Owen Quinn says users should not treat a model's strengths and weaknesses as fixed, since updates can remove old problems and bring new ones to the surface.

Does this mean fake citations are gone?

Not in practice. Columbia University researcher Maxim Topaz points out that false AI citations found in academic papers have risen sharply since 2023. More researchers use AI, and not everyone uses the newest model.

There are caveats to the study itself. It is a blog post from the team, not a peer-reviewed paper. And a flaw missing in a test does not mean it is gone from every question or setting.

Apps worth a look

SUMO: the number puzzle

Tap the right tiles, add them up, hit the target. Easy to learn, hard to master. Free, for all ages.

Tags:stanfordai sycophancyllmai hallucinationsai research

More from this day

See all →

More in: AI

See all →