Wes Bos scanned his office with an iPhone, had Codex rebuild it in Three.js, then made every filament spool throwable

Play the video on YouTube

“Then, naturally, I added physics so I could throw everything around. And a Nintendo Zapper.” That’s how Wes Bos describes the second half of his project, after the part where he turned his office into code.

The phone doesn’t hand over a finished room. It hands over evidence. Apple’s RoomPlan gives “walls, doors, windows and rough furniture boxes,” the LiDAR gives a surface mesh, and the photos carry “materials, labels, cables, shelf contents.” Each photo also records where the phone was and which way it pointed.

That last part is the trick. “The photos aren’t just a folder of unrelated images,” Wes writes. Because every shot has its own camera pose, the agent can render the 3D scene from that exact viewpoint and set it beside the real photo. His video’s description says he stitched it together “with GPT-6 Astra in Codex,” and tags @OpenAI and #ChatGPT_Partner.

Then it loops. Build, screenshot, compare, repeat. “Is the desk too wide? Is that shelf in the wrong place? Does the chair actually look like my chair?” Wes says the layout was one pass, and the materials, lighting, shadows and small details were “a whole other job.”

The playground is where it stops being a floor plan. The room runs on the Rapier physics engine, the shelves stay bolted down, and the code says “every original and added spool/bin detaches independently” from the filament rack. Each crash plays a synthesized “thunk” whose brightness follows the surface and size, and whose loudness follows the force.

The video’s title says he did his “entire house.” The repo ships only the office: “No other rooms, connected-house layout, original scan bundles, or development Git history are included.” And the iPhone app is source-only for now, built in Xcode, with the App Store release still “a separate step.”

Scanning a room into something useful used to mean paying for a service and getting back a model you couldn’t edit. A LiDAR phone plus an agent that checks its own work against photos turns that into a weekend project with code at the end. Wes’s version is a toy office, but the same loop fits an insurance inventory, a server closet, or a rack you need to explain to the person on the other end of the phone.

The open question is how far the loop goes without Wes in it. He says the measurements “are still worth checking against the real thing,” and the tools stop at the side-by-side. The next version of this probably gets judged on who decides when the desk is the right width.