Nesformer Compiles an NES CPU Into a Qwen3.5 Model's Weights, and Super Mario Bros. Runs on Them at One Frame Every Few Seconds

The checkpoint loads with the standard Qwen3_5ForCausalLM.from_pretrained call and needs no custom code. It has 466.6 million parameters in bf16, and the README says “9.3M are nonzero, so the weights zip to 3.3 MB.” Seven Gated DeltaNet layers hold the circuit, plus one all-zero attention layer that’s only there because Hugging Face’s cache expects one.

Inside those layers is a whole 6502: “all 151 official opcodes, every addressing mode, flags, stack, BRK and NMI entry.” What isn’t in the weights is the picture. A small host program “plays the rest of the console around it: it draws the picture, reads the controller, and fires the vblank interrupt.” The host never sees the CPU’s registers or RAM except through the trace the model writes.

The memory is the clever part, and it’s borrowed. nesformer builds on Qwenputer by Divyansh Agrawal, which compiled a RISC-V CPU into Qwen3.5-4B, and reuses its compiler and “central trick: the model’s Gated DeltaNet state is the machine’s memory.” Each DeltaNet head keeps a 128 by 128 state matrix. With the gate held open and a one-hot key, an update “overwrites exactly one row of that matrix, and a one-hot query reads it back. So every row is a memory cell.” The cartridge, RAM and registers all live in rows.

The output reads like a disassembler. Each instruction becomes a short record of tokens, LDX #$40 comes out as X($40 c0 v0 px0 L2), and “real game code averages 1.1–1.3 tokens per instruction.” The program counter isn’t even written down; it’s the last jump target plus a count of instruction lengths since, tallied by three DeltaNet counters.

It’s slow, and the README is upfront about it. The fast engine ran “about 1,650 tokens/s on one CPU core,” and “A real NES would need about 700,000 tokens per second.” A frame takes 1 to 6 seconds, so ten seconds of NES time takes around 25 minutes. When a game sits in its wait-for-vblank loop, the host notices the loop body repeating with no writes and skips ahead, which cuts tokens 2.6 to 4.2 times.

It’s also exact. The weights, free-running, produced 1,916,527 tokens of Thwaite and “all 300 saved frames match” a conventional emulator; Full Game ran 2,645,885 tokens with all 210 saved frames matching. The limits are cartridge-shaped: NROM boards only, official opcodes only, no IRQs and no sound. Super Mario Bros., Donkey Kong and Excitebike qualify. Anything with a mapper chip stops with a message, and the README’s plan for fixing that means storing the whole ROM in the state and adding one more stage, which nobody has built yet.

Most of what we argue about with language models is fuzzy: whether it understood, whether it hallucinated, whether it’ll do it again. This project has none of that. Every token is either the token a 6502 would produce or a bug, and the tests check millions of them. It’s a useful reminder that a transformer is a very large, very general piece of arithmetic, and that “it predicts the next token” can describe a CPU stepping through Super Mario Bros. as accurately as it describes autocomplete. Whether anyone wants an NES that runs at a frame every few seconds is beside the point. The interesting question is what else fits in those 128 by 128 rows.