audio-ai-hub: a JSON-backed paper index for audio AI, and the parts it does not cover
The hub for audio AI research: papers, open models, benchmarks & datasets across audio LLMs, speech recognition, TTS, music & audio generation.
At a glance
- What is it?
- BinWang28/audio-ai-hub is a curated index of 129 audio AI papers, models, benchmarks and datasets across 11 categories, driven by per-entry JSON files and a Python generator script. It is a discovery tool, not a library, and its value depends entirely on how its entries are maintained.
- Who is it for?
- Adopt audio-ai-hub if you need a fast orientation pass over audio LLMs, ASR, TTS and audio generation before committing to a deeper literature search, and if you are willing to open the live site rather than read the README. Do not adopt it as a dependency, an API, or a source of benchmark numbers: it stores links and short descriptions, not results.
- Can I use it commercially?
- Not without permission. GitHub finds no licence file in the repository, and without a licence all rights are reserved by default: you may read the code but not reuse it. Check the README, or ask the authors, before using it.
- Is it still maintained?
- Yes. The repository last received commits 1 day ago.
- What is it written in?
- Mainly Python, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
The problem audio-ai-hub actually addresses
Audio AI research is spread across venues that do not share a taxonomy. A speech recognition paper, a text-to-speech foundation model, a music generation transformer and an audio language model benchmark sit in different arXiv categories, different conference tracks and different GitHub organisations. Someone starting a project in this space has to assemble a reading list from scratch, and the list goes stale within months. audio-ai-hub is a response to that: a single repository that holds 129 entries across 11 categories, from Model and Methods (60 entries) down to Dataset Resource, Safety and Chatbot (3 each). The README states the scope directly: "Curated papers, open models, benchmarks and datasets across audio LLMs · speech recognition · speech synthesis · music & audio generation." The intended reader is an engineer or researcher who wants to know what exists in a subfield before reading any single paper in depth. It is not a package, not an evaluation harness, and not a source of reproducible numbers. It is a map, and the map's usefulness is bounded by how often someone updates it.
How the index is built: JSON entries, a Python generator, and a Pages site
The contribution path in the README describes the mechanism precisely: add an items/<Abbreviation>.json file following the template in schema.json, then run python3 format_input.py to regenerate the README and the site data, then open a pull request. That means the README is not hand-edited prose. It is generated output, and the same generator produces the data behind the GitHub Pages site at binwang28.github.io/audio-ai-hub. The README says as much, pointing readers to the live site for "search, filtering by category and sorting by stars or date" and calling the live site "much faster than scrolling this README." Two other pieces are visible in the repository layout. A workflow at .github/workflows/refresh-stars.yml refreshes the Featured table weekly, which is why the README can describe that table as "Top 8 by GitHub stars" without a manual edit. And CI is described as validating JSON, checking README sync, and rebuilding the site on merge. The data flow is therefore one-directional: JSON in, generated README and site data out, with CI acting as the gate that rejects a PR whose generated artifacts do not match its source.
What the Featured table tells you, and what it hides
The Featured table ranks eight projects by GitHub stars, refreshed weekly. Whisper leads at 108k+, followed by VoxCPM2, MMS, MiniCPM-o, IndexTTS, MusicGen, AudioGen and CosyVoice 3. Star count is a popularity signal, not a quality signal, and the table makes no claim beyond ordering. The more interesting detail is what each row links to: every Featured entry points at an arXiv abstract page, not a repository. So the table answers "what is widely known in this space" rather than "what can I install today." A reader who wants the code has to take a second step. The table is also a snapshot of one ranking criterion. A recent, well-engineered model with a small following will not appear there, even though the Recently added list may include it. The two lists serve different purposes: Featured is a popularity prior, Recently added is a recency feed. Neither is a recommendation, and the README does not present them as one.
The category split is lopsided, and that is informative
Model and Methods holds 60 of the 129 entries, roughly 47 percent. Benchmark holds 19, Speech Synthesis 13, Audio Generation 9. Speech Recognition has 6, Multimodal 6, Survey 4, and Dataset Resource, Study, Safety and Chatbot have 3 each. A reader scanning for ASR coverage will find far less than the topic list at the top of the README might suggest. The category names also overlap in ways the counts do not resolve: a speech synthesis model is also a model and method, and the README does not state the rule that decides which category an entry lands in. That ambiguity is a maintenance risk, because two contributors can classify the same paper differently and the schema does not visibly enforce a single answer. For a reader, the practical consequence is that category filtering on the live site narrows the field but does not partition it. You should expect to check more than one category for a given topic, and to treat the counts as approximate rather than exhaustive.
Where the hub stops being the right tool
Three limits are visible from the material. First, the hub stores no benchmark results. An entry is a title, a date, a category and a short description, plus a link. If you need Word Error Rate comparisons, MOS scores or inference latency, the hub can point you at the paper but cannot answer the question, and nothing in the README suggests it intends to. Second, the licence is unknown. The repository metadata does not state one, and the README does not mention licensing for the index itself. That matters if you plan to mirror the data, republish the tables, or fold entries into another catalogue. The README does provide a CITATION.cff and asks that the hub be cited, which is a request about attribution, not a grant of rights. Third, the hub is a moving target with no releases. No releases were retrieved, so there is no version number to pin, no changelog to diff, and no stable artifact to depend on. If your workflow needs a frozen snapshot of the index, you have to take one yourself by checking out a commit. The repository is not archived, and the last push is dated 2026-09-07, so it is active, but active is not the same as versioned.
What it does that a general paper search does not
The closest alternative is an arXiv category feed or a keyword search on a general paper index. The difference is in the unit of curation. A search returns everything matching a query, ranked by relevance or date, with no domain judgement applied. audio-ai-hub returns a fixed set of 129 entries that someone decided belonged in an audio AI catalogue, each with a one-line description written for this context. The README's own framing is that it is a hub rather than a feed, and the contribution path supports that: a human writes items/<Abbreviation>.json, and CI checks it against schema.json before it lands. That human filter is the product. It is also the cost. A search index updates continuously and covers everything; audio-ai-hub updates when a contributor opens a pull request, and covers only what that contributor thought to add. For breadth, use a search engine. For a pre-filtered starting set with consistent metadata and a stable category structure, the hub is doing work a search engine does not attempt. The two are complements, and the README treats them that way by linking every entry out to arXiv.
Contributing, maintenance and what a PR actually costs
The contribution loop is short and mechanical: create items/<Abbreviation>.json from the schema.json template, run python3 format_input.py, open a pull request. CI then validates the JSON, checks that the README is in sync with the entries, and rebuilds the site on merge. For a contributor, the real cost is not writing the entry. It is running the generator and committing the regenerated README alongside the new JSON, because CI will reject a PR where the two disagree. That is a deliberate design choice: it keeps the README and the site derived from one source of truth, at the price of a noisier diff on every content change. The README also offers a lower-effort route through an issue form titled "Suggest a paper," which routes the addition to a maintainer. Maintenance burden for the project itself is mostly the weekly star refresh workflow and whatever the site build requires. For a user, the cost is zero until you decide to mirror the data, at which point the absent licence becomes the blocking question and the absence of releases means you pin a commit hash rather than a tag.
Who should use it, and what to check before you do
Use it if you are scoping a project in audio LLMs, ASR, TTS or audio generation and want a pre-filtered list with consistent categories before you start reading. Use the live site, not the README, for anything beyond a glance, since the README itself says search and sorting live there. Skip it if you need benchmark numbers, a stable versioned dataset, or a clearly licensed corpus to redistribute. Skip it too if your subfield is Speech Recognition specifically: 6 entries across a topic the README lists in its first line is thin coverage, and a targeted arXiv search will beat it. Before depending on it, verify three things. Open schema.json and one items/<Abbreviation>.json file to confirm the entry format matches what you need. Run python3 format_input.py on a clean checkout and confirm the README regenerates without a diff, which tells you the generator is deterministic and the repository is in sync. And check whether the category counts on the live site match the table in the README, because both are generated from the same source and any mismatch means the merge pipeline has fallen behind.
Editorial conclusion
Adopt audio-ai-hub if you need a fast orientation pass over audio LLMs, ASR, TTS and audio generation before committing to a deeper literature search, and if you are willing to open the live site rather than read the README. Do not adopt it as a dependency, an API, or a source of benchmark numbers: it stores links and short descriptions, not results. Before relying on it, open an items/<Abbreviation>.json file against schema.json, run python3 format_input.py on a clean checkout to confirm the README and site data regenerate without diff, and check the category counts on the live site against the table in the README, since those two are generated from the same source and any drift tells you the merge pipeline is lagging.
Community notes