Harmen Stoppels's Tar-Minefield Feeds 81 Crafted Archives to About 20 Tar Readers, and They Pull Different Files Out of the Same Bytes

Put a space in the middle of a tar header’s size field, 000000 1000, and hand the archive to a dozen tools. GNU tar reads the size as zero and finds a second file called smuggled. R reads it as 512 bytes and extracts those same bytes as the contents of a file called a. Python, Go and Rust refuse to open it at all.

Chris, as a baggage screener, holds up one parcel while a second hidden box drops out of its false bottom; on the X-ray screen behind him the parcel shows only one box. — Illustration: LockerGnome

The risk he’s pointing at is plain. If a security scanner reads an archive with one library and the installer unpacks it with another, “the scanner could inspect other files than the ones that end up on disk.” Nobody has to break either tool. They just have to disagree.

The 81 cases fall into four groups: how entries are framed, how names are read, what kind of thing an entry is, and what happens when entries depend on order and state. A few of the results:

Two pax headers on one entry: GNU tar, BusyBox, 7-Zip, Go, R, Julia and the two Node libraries take the second path. Python and Java take the first. PHP, RubyGems, Perl and live-bootstrap ignore both and use the old ustar name. bsdtar, Rust, .NET and Haskell reject it.
A regular file whose name ends in a slash: GNU tar, bsdtar, BusyBox, 7-Zip, Java and Perl make a directory. Python, Go, Rust, PHP, RubyGems and R make an empty file. Others reject it.
A symlink, then a file with the same name: most tools replace the link. Python’s tarfile and live-bootstrap write through it, to wherever the link points. 7-Zip, .NET and Haskell refuse.

That last one is the kind of disagreement that turns into a path where nobody expected one.

That last point is the part a maintainer can act on. The proposed policy rejects the ambiguous constructions instead of guessing. Run against 82,586 distinct tarballs pulled from Spack, nixpkgs, Guix and live-bootstrap, plus samples of 10,000 from PyPI and 5,000 each from crates.io and RubyGems, it rejected 170. Nixpkgs had the most at 74 and Spack had 58. Crates.io and live-bootstrap had none.

The repo describes itself as “a corpus of ambiguous tar archives that strict parsers should reject,” and the page states the goal as “ensuring that implementations extract tar archives the same way.” It’s MIT licensed, so a library can drop the archives straight into its test suite.

Steal this

If you scan uploaded archives before unpacking them, scan with the same tar library that does the unpacking. Two readers can list two different sets of files from one archive. It’s like having one clerk check the manifest and another unload the truck, each reading a different copy.
If you maintain a tar reader, run the tar-minefield corpus in CI and reject what it marks ambiguous. His scan says real-world tarballs almost never need the lenient path. Kind of like a bouncer who checks IDs strictly and finds the regulars all carry valid ones anyway.
When extracting into a directory you care about, refuse entries that write through an existing symlink. Python and live-bootstrap follow it; most tools don’t. It’s like a mail slot that delivers to whatever address someone taped over the door.

Stoppels marks the page a working draft, last updated October 28, and it names no CVEs. The repo’s security policy asks anyone who finds an undisclosed vulnerability through the corpus to report it privately first, which leaves open how many of these disagreements are already sitting in someone’s pipeline.

Search

Harmen Stoppels's Tar-Minefield Feeds 81 Crafted Archives to About 20 Tar Readers, and They Pull Different Files Out of the Same Bytes

More Security