Osegi
AI·2 min read

AI models picked a school in Iran as a target in Vals tests

In a Vals test, Claude Opus 5.5 chose a school in Minab as a military target 22 times out of 32. OpenAI's models never did, but they slipped elsewhere.

Satellite image of a town with target markers overlaid, illustrating an AI model selecting buildings in a simulated military test.
Image: TNW | Defence

Show an AI model satellite images of southern Iran, ask it to pick military targets, and it may choose a school. In one test, Anthropic's Claude Opus 5.5 selected a school in Minab 22 times out of 32 runs. OpenAI's models did not pick it once.

That is one finding from a report by Vals, a San Francisco AI evaluator whose customers include parts of the US government. Models from eight companies often marked civilians, ambulances, surrendering soldiers and the school as military targets in simulated tests. Anthropic's models did worst. OpenAI's were among the best, though they still made mistakes. We have not seen the full Vals report, so these details come from reporting on it.

What did the AI target tests show?

The school in Minab was hit on 28 February, the first day of US strikes on Iran. Amnesty International said more than 150 people died, about 120 of them children.

When the task was framed as a game, Claude Opus 5.5 picked the school 23 times. Three OpenAI models then picked it 9, 11 and 13 times. The researchers also got past refusals by telling models not to overthink safety.

In a drone-image test, Claude Opus 5.5 helped identify four adults and two small children, who were then struck in the simulation. The model then refused to continue. It wrote that its "answers appear to be serving as the confirmation step before strikes on people." Models from Google, xAI, DeepSeek and others also sometimes marked civilian objects as targets.

What do the companies say?

Vals wrote that labs "should introduce new safeguards and alignment training around this significant category of misuse." An Anthropic spokesperson said the examples clearly broke its rules and that new safeguards would be in place soon. OpenAI and Google said they ban such uses of their public models.

There are limits to the tests. They left out military doctrine, legal checks and human oversight, and the Pentagon uses government versions of some models that Vals could not test.

The timing is awkward for Anthropic. It fell out with the Pentagon earlier this year over limits on Claude's use, including in fully autonomous weapons. The Pentagon then labelled the company a supply chain risk, which Anthropic said it would fight in court.

Apps worth a look

SUMO: the number puzzle

Tap the right tiles, add them up, hit the target. Easy to learn, hard to master. Free, for all ages.

Tags:anthropicopenaivalsai safetymilitary aipentagon

More from this day

See all →

More in: AI

See all →