earshot gives coding agents ears: describe, match and critique sound

Coding agents can render a film but cannot hear what they made

The premise behind earshot is a specific gap. AI coding agents can now write the code that renders a film, a product demo or a game, but the documentation argues they still cannot hear the result: they cannot tell a premium key press from a cheap one, notice that a hit lands 80 ms after the cut, or hear that the music swells in the wrong place.

earshot answers that by turning sound into evidence an agent can reason about. The output is not a waveform picture for a human to interpret. It is text: what a file sounds like in words, how well a description fits, the best moment inside a long recording, whether a sound is a real recording or code-synthesized, a second-by-second listening transcript of a film, an annotated picture of that film, and a scorecard against professional launch films.

The audience is an agent workflow, not an audio engineer at a desk. Everything runs locally on CPU, and the tool is offered in three shapes: a CLI, a Python API and an MCP server for Claude Code and other agents.

Twenty commands, each returning a plain-language summary and a JSON body

The command surface is wide because the question an agent asks about a file can be narrow. describe returns what a file sounds like, with calibrated tags, the closest alternative readings, AudioSet labels and measurements. match scores how well a description fits a file, with a cosine value, a z score among the vocabulary and a verdict of strong, plausible, weak or no match. find returns the best moments inside a long recording as start and end seconds, and rank sorts many candidate files against one description.

Other commands answer different questions. realness decides whether a file is a real or professionally produced recording or something code-synthesized, with evidence and per-window scores. timeline produces the listening transcript with events, sections, loudness, a beat grid and the sync of sound to picture cuts. sheet writes an annotated PNG with frames, labelled events, a spectrogram and a waveform. critique scores a launch video against a profile of seven professional launch films and returns timestamped fixes, and profile builds such a profile from references you supply.

The output convention is uniform, which matters more for agent use than the individual features. Every command prints a plain-language summary first, and the full result is available with --json or saved to a file with --out. Errors are one line on stderr in the form earshot <command>: error: ... and the process exits with 1.

Each CLI call reloads the models, so the MCP server is the real interface

Here is the operational catch, stated plainly in the documentation. Each CLI call is a new process and loads the models, roughly 20 to 40 seconds, before it answers. The commands measure, sfx and kit load no models at all, so they are fast. For anything beyond a few calls the recommendation is to use the MCP server, where the models stay loaded, or to write one Python script that imports the API.

That single design decision shapes everything else about using the tool. An agent that shells out to the CLI for every question pays the load every time. An agent that connects to the MCP server pays it once for the session. The Python route, from earshot import api, is the option for a script that asks many questions about many files in one go.

The same reasoning explains why the tool is offered three ways at all. The CLI is for a human at a terminal or a one-off check, the MCP server is for an agent session, and the Python API is for batch work where process startup and model loading dominate the cost.

The forge critic grades renders on tone, envelope, artifacts and polish

earshot also goes in the other direction. The agent writes a sound as code, a recipe of physical synthesis built from modal resonators, contact excitations, filtered noise, a computed room and a microphone, with no recorded audio inside. The forge renders candidates from the recipe parameters, and a critic built from earshot scores each one.

The critic works in two layers. The learned layer checks description match against named confusions, realness, closeness to real reference recordings and measured shape. The model-free layer applies penalties: one third-octave tonal balance, the spectro-temporal envelope, where a tail that dies too fast shows up, artifacts including clicks, aliasing, DC and infrasound, harsh resonances, comb ring, cut tails and clipping, and polish factors such as attack, onset snap and tail smoothness.

Ordering is deliberate. A cheap model-free pre-screen ranks many renders using the same measurements, so the heavier models only hear the best candidates, and an evolutionary search keeps the best and mutates them. The agent then reads the critic in words, for example that something sounds like a cork pop or a glass ting, or that the decay falls outside the references, looks at spectrograms beside the references, rewrites the recipe and runs it again. A command line shows the shape of that loop:

earshot forge --list
earshot forge keypress --refs path/to/real/keypresses/ --budget 144 --variants 4 --dir made/keypress
earshot forge whoosh --refset whoosh_fast --budget 96 --dir made/whoosh

A bank of 201 code-synthesised effects, aimed at the picture

Version 0.3 added the part aimed at video. The project ships a bank of 201 sound effects stored as code, which render by id in seconds with no models and no network involved. The sfx command aims a whoosh, hit, click or note at a moment in the picture, with duration, apex and pitch controls, and it audits what it renders. sfx list and sfx check do not use models at all.

The kit command works at film level. A film gets a sound kit, one bank sound per role, rendered into cached takes with measured onset, peak and end points, and kit check verifies the result without models. The point of measuring those three numbers is sync: when a sound effect must land on a cut, you want to know where its onset actually is rather than where you asked it to be.

This is the part of earshot that can run on a machine with no model download at all. Everything in the sfx and kit path is synthesis from code, which also means the results are reproducible, unlike a library of downloaded recordings.

Running the first analysis: describe, match, find and critique

The package is built with setuptools, requires Python 3.10 or newer and installs a single console entry point named earshot. The dependencies are scientific Python plus torch and transformers, and the project file flags that the torch wheels should be the CPU ones:

pip install torch --index-url https://download.pytorch.org/whl/cpu

Optional extras exist for the MCP server, for the optional audio-LLM path and for tests, named mcp, llm and dev. After installation, earshot doctor checks the install, the models, the data files and the CPU settings, and doctor --load goes further and loads the models.

A first real use is four commands on files you already have:

earshot describe enter.ogg
earshot match key.ogg "a deep thocky mechanical keyboard press" --alts "a clicky blue switch keyboard" "a mouse click"
earshot find keys.ogg "a single mechanical key press" --top 4
earshot critique film.mp4

The first tells you what the file sounds like, the second answers whether a description fits and against which alternatives, the third returns the best moments in a long recording as trims, and the fourth produces the launch-video scorecard. If you only want measurements, measure gives you LUFS, true peak, LRA, envelope, spectrum, tempo and key without touching a model.

An alpha package with a two week release history

The project is young. The first tagged release, v0.1.0, carries the 22 second launch film in which every sound was made in code by earshot forge with no recordings and no music track, and it was published on 2026-10-15. v0.2.0 followed on 2026-10-16, the last push to the main branch was on 2026-10-18, and the version recorded in pyproject.toml is 0.3.0, with the release history still trailing the file.

The classifier in the project metadata says Development Status :: 3 - Alpha, and the repository layout is consistent with that: the earshot/ package, docs/, eval/, skills/, tests/ and a CHANGELOG.md, with no separate site beyond the GitHub repository. The licence is MIT, so the code can be reused and modified without restriction beyond the licence terms themselves.

One command deserves a separate note. llm FILE --yes-upload performs an optional audio-LLM critique with Gemini, and it is only performed with your own key and explicit consent. That is the opposite design to the rest of the tool, which runs locally on CPU, and it is the one path where audio leaves the machine.

Editorial conclusion

Use earshot when an agent has to make a decision about audio it cannot hear, or when a sound has to be produced from code rather than from a recording. Do not expect it to be a music editor, and do not script it as if each call were cheap, because the documented 20 to 40 second model load applies to every command except measure, sfx and kit. Before building on it, run earshot doctor to check the install and the model files, and read CHANGELOG.md, since the project is tagged as alpha and still changing.

Frequently asked questions

Official sources

1. Issues
2. Leonxlnx/earshot on GitHub
3. License: MIT
4. README
5. Releases