AIworld-modelsraspberry-pipico-2-wrustreinforcement-learning
tinyworldformer Runs AI-Dreamed Pac-Man, Breakout and Doom-Lite on a Raspberry Pi Pico 2 W, and Its Dream-Trained Player Scored 210 in a Game That Tops Out at 56
The open-source tinyworldformer runs two tiny neural networks on a Raspberry Pi Pico 2 W, one generating each game frame and one playing it, and agents trained only inside that dream scored 210 where the real game tops out at 56.
The idea is a game with no game code running. The README puts it plainly: the first network “creates the game: given the current frame and the action, it predicts the next frame and the reward. The second one plays that game.” On the chip, “the actor only sees frames the world model, and the world model only gets actions from the actor.”
Five games come in the box: Breakout, Doom-lite, Flappy Bird, Pong and Pac-Man lite. The Doom-lite is a raycast first-person view of an 8-by-8 maze with an enemy that “walks shortest paths to you.” Pac-Man lite has a 23-by-21 maze, 249 pellets and two ghosts. You can grab the controls whenever you like: “any button press hands control to you, so you can play the dream yourself.”
The developer, SheepSeb (Sebastian-Ion Nae on GitHub), says the project started with a memory question. He got interested in looped transformers, “where one transformer block is applied several times with the same weights instead of stacking different layers.” Reusing weights is cheap on memory, so he “wanted to see if a looped world model could be small enough to run on a microcontroller.”
It is. The world models run 26k to 131k parameters in int8, the actor 9k to 16k in float32. Breakout fits in 361 KB of flash and 299 KB of SRAM; “Pac-Man, the largest, uses 600 KB and 460 KB.” Nothing is allocated at run time. Core 1 of the RP2350 dreams the game while core 0 handles USB and the optional screen, and the int8 math leans on an instruction that does “two multiply-adds in one instruction.”
Then the player started cheating. A neural network that imagines a game makes mistakes, and an agent trained inside that imagination finds them. One Breakout agent in the dream “got 154 points with 26 bricks left, because the world model didn’t remove bricks from the corner when it were hit.” Doom-lite agents “learned to snipe from 5 cells away.”
The headline number came from the same bug class. “Agents trained only in the dream scored 210 there (the real maximum is 56) and 28 in the real game,” the README says, “as World Models (Ha & Schmidhuber) predicts.” The fix was to collect real games played by the cheating agent and fine-tune the world model on them: “Each collect-and-fine-tune round closed the exploit that agent had found.”
The dream isn’t perfect yet. In a 5,000-step check, the Pac-Man world model drew 742 wrong game states and paid 38 wrong rewards. And one basic number is missing. The project write-up says a desktop CPU dreams a frame in 0.7 to 1.5 ms, then admits: “I haven’t measured the frame rate on the Pico itself yet.”
Small world models running on cheap boards look likely to become a common hobby benchmark, the way Doom ports are now, because the gap between dream score and real score is easy to show and hard to fake. The more interesting test is still ahead: how fast the Pico dreams a frame, which nobody has measured yet.