Pokémon Red and Blue, the iconic Game Boy games that defined an era, are now hosting something completely new—state-of-the-art artificial intelligence. In a surprising mix of old-school gaming and new-age research, massive language models such as Google’s Gemini and Anthropic’s Claude are being tested in a very public manner. The AI models are being made to play through the classic Pokémon games live on Twitch, with gamers tuning in—and responding—to their every action.
The Emergence of AI-Augmented Pokémon Playthroughs
What began as an entertaining, quirky experiment has evolved into a legitimate, albeit unorthodox, measure of AI capability. Anthropic’s Claude initially broke the headlines, incrementally working its way through Pokémon Red while demonstrating its capacity to deal with open-ended tasks and remain concentrated over extended periods. Next was Google’s Gemini 2.5 Pro, which jumped into the fray thanks to a livestream hosted by independent software engineer Joel Z. He aimed to see just how far Gemini could progress through the game.
The outcome? A weirdly engrossing spectacle. Seeing an AI struggle through the same puzzles and problems that used to confound six-year-olds in the ’90s is entertaining and strangely enlightening. The streams have become events not to be missed by both AI fans and throwback gamers, providing insights into how language models have progressed—and how much further they have to go.
How AI Models Play Pokémon: The Agent Harness
Unlike human players, these AI models aren’t cradling Game Boys. They work through something known as an agent harness—a system that provides the AI game with screenshots, overlays with helpful data, a window into the game’s memory, and software to take text-based choices and turn them into button presses. Some harnesses have even added map-navigating tools or puzzle-solving tools.
This structure is more than beneficial—it’s necessary. How the game is structured to present to the AI, what tools are on hand, and to what extent developers intervene all contribute greatly to the performance of the AI. It also makes it so that it can be difficult to directly compare various models of AI, as no two harnesses are the same.
Comparing Gemini and Claude: Not So Simple
By appearances, Gemini appears to have taken the lead. As of June 2025, it had completed Pokémon Blue, earning eight badges and beating the Elite Four. Claude, however, hadn’t even made it to the third badge in Pokémon Red. Google commemorated the achievement, with CEO Sundar Pichai sharing about the feat.
But the full story is more complicated. Joel Z, who runs Gemini’s stream, and David Hershey, who manages Claude’s, both emphasize that these are very different experiments. Each model uses its custom harness, and the level of human intervention varies. Joel has been open about occasionally tweaking Gemini’s setup or stepping in to fix issues, but says he never outright solves the game for the model.
The Human Element: Developers and Interventions
There’s no getting around the fact that human support plays a big role. Both Gemini and Claude have benefited from developer interventions, whether it’s fixing bugs, adjusting the harness, or offering nudges when the AI gets stuck. Joel describes his role as helping improve Gemini’s reasoning rather than handing it solutions. One anecdote: when a recognized bug was hindering advancement, he instructed Gemini to address a Rocket Grunt twice—a band-aid that echoed how the game was patched in future versions.
These patches, behind the scenes, aim at a larger problem. Current AI models are capable, yet they have difficulty with sustained, multi-step problem solving. Without some assistance, even top models may stall in loops or freeze when encountering an unforeseen impasse.
What Pokémon Teaches Us About AI Reasoning
Pokémon can be a simple game, but for an AI, it is a maze of complexity. The goals are easy enough to understand, the rules are simple, but the world of the game is fraught with distractions and implicit rules. Short-term tasks such as battling or selecting items are possible for Gemini and Claude, but they flounder at higher-level objectives such as long-term planning or recovering from setbacks.
For researchers, it’s not so much a matter of whether the AI can defeat the game. The real question is whether it can play in a manner that makes sense to a human observer. It’s easy enough to brute-force an obstacle—another thing entirely to foresee, strategize, and respond to problems, like a master player.
Why Watching AI Play Pokémon Is So Addictive
It’s oddly endearing to see an AI stumble over the same obstacles we used to when we were children. The Twitch chat throughout these streams is filled with cheers, gasping, and nostalgia. Fans are not only watching to see AI perform, but also to experience a part of their childhood from a completely different perspective.
Part of the wonder. Pokémon is a cultural reference point, and watching it being used to verify artificial intelligence closes the gap between yesterday and tomorrow. It makes us remember how far technology has advanced—and how endearingly human it remains when it attempts to play a child’s game and gets stuck in a cave.
These tests can be technological show-offs, but at their core, they’re about the relationship between retro gamers and new-school hardware, between the past and the future, and between what AI is and what it could be.
