Osegi
AI·2 min read

Idea Lens: a detector that checks if the idea came from AI

Idea Lens ignores word choice and looks at how a text argues. Researchers say it can tell whether the idea behind a text came from a person or a chatbot.

A researcher reading a printed scientific paper with highlighted passages next to a laptop showing an analysis of the text's argument structure.
Image: t3n Magazin

Most AI detectors ask who wrote the sentences. Idea Lens asks who had the idea. Researchers from the University of Maryland and Google Deepmind built it to spot AI involvement in the argument itself, not just in the wording.

The work is a preprint, so independent experts have not reviewed it yet. Tools like Pangram look at surface features: word choice, sentence structure, statistical patterns in the prose. Idea Lens instead splits a text, say a scientific study, into segments and describes what each one says and what it does, for example whether it is evidence, a recommendation or a description. A separate language model then rewrites the content, which strips out the author's style and word choice. A side effect: it works across many languages.

How does Idea Lens tell human ideas from AI ideas?

It reads the order and type of arguments. People and language models both arrange content and function, but models fall back on similar patterns. According to the team, human texts contain more specific details and instructions that back up the main point. AI text is more generic, with interchangeable details. People also name limits and hurdles more often, while AI tends to skip or underrate drawbacks and trade-offs.

The team trained it on about 800,000 texts labelled as AI or human by Pangram. Pangram is not perfect, which the researchers see as useful noise in the training data. Testing included texts from before ChatGPT.

The gains show up with mixed authorship. When AI helped write texts built on human ideas, Pangram 4 flagged the same texts as AI-generated up to 92.4 percent of the time. Idea Lens dropped that to as low as 7 percent, depending on the text's structure. When humans wrote texts on an AI-generated structure, Idea Lens spotted the AI origin in 68 percent of cases. Prose detectors managed under 10 percent.

Where does it fall short?

Independent researchers say it does not suit true co-authorship, where human and machine shape ideas together and the line disappears. It is also a black box: it is unclear which cues it uses and how it weighs them, as the team around Rishanth Rajendhran admits.

Rajendhran told Science the tool should be combined with other detectors and applied to a researcher's whole body of work, not single cases, given the reputational damage a false accusation causes. The team also sees it as a way for scientists to check what role AI plays in their own work. There is a website where you can try it.

Apps worth a look

SUMO: the number puzzle

Tap the right tiles, add them up, hit the target. Easy to learn, hard to master. Free, for all ages.

Tags:idea lensai detectionpangramgoogle deepminduniversity of marylandchatgpt

More from this day

See all →

More in: AI

See all →