Osegi
AI·2 min read

Claude playing Pokémon Red fell off the same ledge three times

Anthropic gave Claude a notes folder and Pokémon Red. One wrong note about a Pokémon Center sent it off the same Mt. Moon ledge three times.

A Game Boy screen showing a Pokémon Red trainer standing at a ledge in front of a cave entrance, with a notepad beside it.
Image: ギズモード・ジャパン

Claude playing Pokémon Red walked off the same ledge near Mt. Moon three times in a row, because it trusted a note it had written itself. The mistake comes from a research post Anthropic published on November 6 about how AI agents handle memory.

In the experiment, Claude got a folder it could read and write between plays. What to save and how to organise it was entirely up to the model.

Here is how it went wrong. Near the Pewter City entrance of Mt. Moon there is a ledge: drop down and you have to backtrack. Claude used Pokémon Centers as landmarks. The one by the Pewter entrance sits to the west, the one nearest the Cerulean entrance to the east. Before going in, Claude wrote that the Center was on the east side. Leaving the Center, it declared Mt. Moon was behind it and updated its note to "Mt. Moon: passed".

Then it fell off the ledge and landed back on the road it had just walked. Its navigation tool does not show ledges, so Claude concluded it had gone too far west and that it was on the right side all along. Repeat. After the third fall it left a note saying, roughly, that it would stop rather than keep looping. An automatic "continue" prompt restarted it, and it headed back to Pewter City to heal and shop.

What is "dreaming" for an AI?

This is the technique Anthropic tested. Humans sort experiences into long-term memory during sleep, and researchers have proposed that AI could tidy its memory between tasks in a similar way. Dreaming has the model review past records and fix mistakes or contradictions.

When it was run on the faulty note, Claude checked it against the session logs, spotted the error and rewrote it. It then found the cave entrance, went in and reached Cerulean City. Attempts that kept falling, and attempts with no notes at all, never got over the mountain.

More notes, fewer badges

The study also compared Sonnet 3.7 and Fable 5 at 10,000 steps. Sonnet 3.7 had written 413 files in 225 folders and had one gym badge. Fable 5 had nine files in two folders plus one "global.md", and four badges. Its notes held strategies and rules it had worked out itself. Opus 5.5 cleared the game with only a small notes folder.

One Fable 5 note, written after seven failed Poké Ball throws at a Magikarp, says to change the timing after three misses in a row by adding an extra wait between menu inputs. It caught the Magikarp right after.

Apps worth a look

SUMO: the number puzzle

Tap the right tiles, add them up, hit the target. Easy to learn, hard to master. Free, for all ages.

Tags:claudeanthropicpokemon redai memorydreamingai agents

More from this day

See all →

More in: AI

See all →