LockerGnomeFor serious tech professionals with a silly side.
Subscribe

Search

Softwarecompressionopen-sourcewebassemblyaistorage

Aqueous, a New Archiver, Undoes JPEG, PNG and ZIP Compression, Squeezes Harder, Then Rebuilds Every Original Bit for Bit With WebAssembly

The open-source aqr tool unpacks files that were already compressed by weaker encoders, recompresses their contents with stronger models, and ships the WebAssembly decoders inside the archive; on a 10 GB benchmark it came in at 2.63 GB against tar plus zstd's 4.68 GB.

Chris unpacks a stack of nested boxes and presses their contents into one small cube
Illustration: LockerGnome

The trick is that every file in an Aqueous archive is a small program. The README calls it “a small stack-machine program over stored blocks,” a formula that “applies WebAssembly modules to undo what the compressor did.” A JPEG goes in, the archiver peels the JPEG coding off, stores the image data in a form its own models squeeze much harder, and writes down the recipe for putting the JPEG back exactly as it was. Exactly means exactly: “rebuilds the original files bit for bit on extraction.”

Because the recipe and its decoders travel together, the reader doesn’t need to know any file formats. “A decompressor needs no knowledge of the formats inside; known modules run natively, unknown ones in a sandbox,” the README says. The 10 GB benchmark archive “includes its decoder modules.”

The format list is long for a project this young: PNG, JPEG, PSD, BMP and DDS textures; MP3, FLAC, WAV and AIFF; PDF and SQLite; gzip, zlib, ZIP, LZ4, LZMA, tar and git checkouts; Windows and Mac executables. Per format, the gains are blunt. A JPEG shrinks to 0.785 of its size where zstd manages 0.913; a PNG goes to 0.802 where zstd barely moves it, at 0.997.

The headline run, on 79,431 files on an Apple M4 dated October 27, took 136 seconds to compress, including a round-trip check of every file, and 19.2 seconds to extract. Plain tar plus zstd took 33 seconds and landed at 4.68 GB; zpaq 7.15, a long-time heavyweight of archive compression, made 3.70 GB. And because the formula indexes each file’s blocks, pulling one file out means “reading little more than its own blocks.”

The author, who goes by lifthrasiir on GitHub, is upfront about how it was built: “This project makes heavy use of LLM-based coding agents,” and “A human set the ideas and the goals; AI did the implementation and the experiments.” The format spec is still “an Editor’s Draft, not yet fixed,” and the per-format numbers are labelled “tentative.” Nobody has yet tried opening one of these archives years from now with a reader that has never seen the formats inside.

The bigger shift is the author’s split of labour: a person setting goals and agents running the experiments, on a problem as unforgiving as bit-exact compression. If that holds up, expect a lot more niche formats to get serious engineering they could never have paid for. The test that matters is the boring one: whether an archive written this month still opens, byte for byte, on a machine nobody has built yet.

More Software

Every Software story →