Ploom Runs Every Thread Order of a Python Test Until One Fails, Then Saves That Order So the Bug Replays on Demand

Instruction-manual cutaway of Chris locking a cash drawer as two strangers reach for the same stack of bills from opposite sides — Illustration: LockerGnome

Two people share a bank account holding 100. Both withdraw 100 at the same moment. Only one of them should get the money.

That’s the example in Adil Zouitine’s README, and it’s the race every concurrency course warns about. Each thread checks the balance, sees 100, and withdraws. Written with plain threading.Thread, the README says, “this code passes almost every time,” which means the test goes green on your laptop and red on the build server once a month.

Swap in ploom’s own Thread and SharedCell, call ploom.check, and the tool “finds the bad one in a few seconds.” The failure comes back as the account is overdrawn, with two files beside it: the original trace and a minimized one. The minimized trace reads like a crime scene report. Lucas reads 100. Mathieu reads 100. Mathieu withdraws, balance 0. Lucas withdraws, balance -100.

The bank example ships in the repo without the lock, so you can watch it fail. Put a Lock around the check and the withdrawal, and “now all 127 orders pass.”

The trick is small. Every ploom operation, a lock grab, a queue put, a shared read, is a scheduling point, and only one task runs at a time, “so the result does not depend on the GIL.” A depth-first search picks each runnable task at each point. Each schedule runs in a fresh Python process with the same hash seed, so state from one try never leaks into the next and set order doesn’t shuffle between runs.

Zouitine, whose GitHub profile lists him as a founding research scientist at UMA and formerly of Hugging Face, credits the idea to Loom, the Rust concurrency tester: “Loom showed how to explore schedules and reproduce failures.” Ploom defines its own Python operations and doesn’t copy Loom’s C11 memory model. The repo went up on October 25. What it hasn’t met yet is a real codebase’s thread pool.

Why it matters

A flaky test is a schedule nobody wrote down. The bug was always in there; the operating system just rarely picked the losing order. Recording the order turns “can’t reproduce” into a pytest command ploom prints for you: --ploom-replay=.ploom/ID-original.json.

Minimizing is the part that saves an afternoon. A raw failing schedule can be hundreds of switches long. Ploom looks for the same failure “with fewer preemptions,” a preemption being a switch away from a task that could have kept going, and hands you the four-line version.

It refuses to fake a pass. Search limits raise IncompleteExplorationError, and the README is blunt: “A limit never counts as a pass.” A tool that quietly stops at 10,000 schedules and goes green would be worse than no tool.

Free-threaded Python raises the stakes. Ploom’s scope table lists standard CPython 3.13 and 3.14 plus free-threaded 3.14, the build without the GIL. Code that got away with sloppy sharing because the interpreter serialized it loses that cover there.

Steal this

Model the shared state, then let ploom do the guessing. When a test fails “sometimes” and the suspect is two threads touching one value, rewrite just that scenario with ploom.SharedCell, ploom.Thread and ploom.Lock, and run ploom.check. Kind of like setting a trail camera instead of staking out the yard every night.
Commit the minimized trace with the fix. Once a race is found, keep the -minimized.json and the printed replay command in the bug ticket, so whoever touches that code next can rerun the exact failure. This is like keeping the broken part in a bag next to the repair invoice.
Keep the scenario tiny or cap the switches. Each schedule costs “about 0.1 seconds” in a fresh process, so test two or three threads doing the one risky thing, and for anything bigger set Config(max_preemptions=2). Think of it as checking every key on the ring, not every key in the hardware store.
Your asserts are the detector. Ploom “does not detect data races. Your assertions find the bugs,” so before you run it, write the assert that states the rule, like balance >= 0. Kind of like a smoke alarm with no battery: it visits every room and never makes a sound.

The limits are written down, which is rarer than it should be. Ploom sees only its own operations, not native threading, queue or I/O, and it “does not roll back files, network calls, or other external effects between schedules.” So the next step is the one the documentation is built for: somebody pointing it at a real worker pool, with real queues, and finding out how many of their “flaky” tests were never flaky at all.

The open question is size. A two-thread bank account has 127 orders. A web server’s job queue has more than anyone will wait for, and whether two preemptions are enough to catch the bugs people actually ship is something only a lot of trace files will answer.

More Software