Antithesis's fuzzer ascends NetHack in a median under two minutes and finds that typing ;|: pins a CPU core at 100%

About an hour into one run, 47 of the 48 cores in the test machine were doing nothing. The 48th was at 100%, and it stayed there.

That was the first clue. The authors say the culprit was three keystrokes. Typing ;|: in a new game made the machine hang. Will, one of the post’s three authors, put it this way: “now your computer is a space heater.” (A very hot new feature.) A GDB backtrace led them to a function with this comment in it: /* warning: no bounds checking performed */. Somebody left a warning, and the code obeyed it.

The fuzzer wasn’t hunting for that. The authors write that they “never told the fuzzer where to look.” Antithesis explores a game by feeding it inputs, saving states and branching many timelines from them, and the post describes it as “something akin to a probabilistic programming language”: a person specifies part of the task and search fills in the rest. Their takeaway after finding the freeze was this: “Reasoning, whether human or machine, is great at checking a bug’s vicinity. It’s much worse at telling you where to look.”

A naive fuzzer also wastes time on the dumbest things. Every time it sent V, it had to read through “a 14-page description of NetHack’s history,” so the authors eventually stopped sending the key. A robot that reads the credits at every level is not winning fast.

The early harness was patched into NetHack itself. Later they built a Player workload that turns fuzzer bytes into high-level actions like wielding a weapon. The post says that cut their strategy code from thousands of lines to a short config file. For comparison, it says an earlier bot, BotHack, “needed 16,000 lines of Clojure,” and this setup “ascended with a short config file.”

The results are impressive and the post says where it limps. The first ascension relied on scaffolding that read hidden state: luck, alignment, prayer timeout, the full inventory and the level layouts. The authors say they are working to remove it, and that a “black-box ascension is within reach,” which is not the same as having done it. They also say some runs won on luck, like a lucky polymorph or a trapdoor.

The number deserves its own caution. The headline’s 80% and five minutes are the authors’ own claim. The closest figure in the post’s own trial data is a 79.0% ascension rate across 480 stock-RNG trials, and the post says a patched “well-tempered workload” version reached 93.3% and ran up to three times faster. Stock-RNG runs, it notes, “take longer to find interesting states.”

Other finds: a double free and an assertion failure, one of them reported upstream as NetHack issue #1682. One long-blocking segfault turned out to be a buffer overflow in the authors’ own instrumentation, a thing named _totally_safe_buffer. A reward function overflowed an integer, and past a certain depth, dropping items became the most rewarding action in the game. They got ascensions for every role, including a weaponless vegan Monk.

My guess is the fun part is the pitch, not the bug count. A company that makes testing software picked a game everyone has heard of, with a cult of players who know exactly how hard a win is, because “it beat NetHack” travels faster than “it found a double free.” The same method, told to explore a database or a payment system, is the actual product. Expect other testing vendors to start picking games, too.

The unsettled part is the black-box ascension. Until the scaffolding stops peeking at luck and prayer timeout, the fuzzer is playing with an open book. The authors say it is within reach, so the next version of the post should either show it or explain why not.

The ;|: freeze is still the best part. Three characters, and a roguelike turned into a space heater.