
[Simon Willison] recently used a coding agent in an unusual way. A software system temporarily used the microphones on devices scattered around the room to triangulate the source of a mysterious, intermittent beep coming from within one of the many boxes in the room.
The main part of the system is a Python web server, and the listeners can be any old devices one might have hanging around. Scan a QR code, and the device becomes a listening post by leveraging the Web Audio API. With listeners spread around the room, the system has everything it needs to patiently wait, listen, and zero in on the mystery beep. The culprit? An old carbon monoxide detector, buried in a box and emitting a low battery chirp.
That’s neat, but what this project really demonstrates is the benefit of hacking together a way for a coding agent to actually sense the real world, the better for it to design a solution to a physical problem.
The ad-hoc framework of web-based listeners allowed [Simon] to give the agent the ability to gather audio data on its own to refine and recalibrate its approach during development, without any need for him to record audio and pass it on manually.
It’s a bit reminiscent of this WiFi analyzer connected to an AI model, a project that hints at the possibilities of a physical tool that can be steered by an agent. Check out the GitHub repository for beep-locator if you’d like a closer look at the code itself.
