AudioMuse-AI: Sonic Analysis Playlists for Self-Hosted Music Servers
AudioMuse-AI uses sonic analysis to rediscover forgotten songs, uncover hidden connections in your music library, and generate intelligent playlists for Navidrome, Jellyfin, LMS, Lyrion, Emby and Plex: no metadata or external services required.
At a glance
- What is it?
- AudioMuse-AI analyzes the actual sound of your music library to build playlists and maps without metadata or cloud APIs. It supports Navidrome, Jellyfin, LMS, Lyrion, Emby, and Plex, but its strengths come with real operational trade-offs.
- Who is it for?
- Adopt AudioMuse-AI if you run a self-hosted music server, want playlists based on acoustics rather than tags, and can handle a multi-container Python deployment. Skip it if you need a single binary, rely on metadata-only search, or require a stable plugin API for every server.
- Can I use it commercially?
- Yes, with strict conditions. AGPL-3.0 is a network copyleft licence: if people use a modified version over a network, for example as a hosted service, you must offer them its source code under the same licence.
- Is it still maintained?
- Yes. The repository received new commits within the last day.
- What is it written in?
- Mainly Python, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
The Problem: Playlists That Ignore How Music Sounds
Most self-hosted music servers build playlists from metadata: genre tags, artist names, or user ratings. That approach misses the actual sonic character of a track. Two songs with the same genre tag can sound nothing alike, while a forgotten B-side might share a groove with a current favorite but never appear in the same list. AudioMuse-AI addresses this by analyzing the audio waveform itself, using tools like librosa and CLAP, to create playlists based on tempo, energy, mood, and instrumental texture. The intended user is a homelab owner who has a large local library, runs a media server like Navidrome or Jellyfin, and wants discovery features that do not depend on external APIs or on the quality of existing metadata. The project explicitly targets self-hosters: it can run in Docker Compose, Podman, or Kubernetes, and it supports both AMD64 and ARM64 architectures.
How Sonic Analysis Replaces Metadata
The core mechanism is a pipeline that extracts acoustic features from each audio file and embeds them into a vector space. The README mentions CLAP, an audio-language model, and librosa, a standard Python library for audio analysis. The system also uses ONNX for model inference. Once tracks are embedded, the application performs clustering to group sonically similar songs, which powers features like the Music Map, a 2D visualization of the library based on acoustic distance. The same embeddings support text search: a query like calm piano songs is matched against the audio features, not against tags. The architecture splits into Flask and Worker containers, where the worker likely handles the heavy feature extraction and the Flask container serves the API and UI. Version 3.2.0 moved job queuing to PostgreSQL, removing the Redis dependency, which simplifies deployment. The multiple music server feature, introduced in 3.0.0, lets one AudioMuse-AI instance connect to several servers; duplicate detection ensures a song shared across servers is analyzed only once, and the result is shared across all connected servers.
Getting It Running: Docker, Kubernetes, and Native Apps
The README says you can run AudioMuse-AI locally with Docker Compose or Podman, deploy it in a Kubernetes cluster using a Helm chart from a separate repository, or use native applications for macOS, Windows, and Linux. The repository includes a deployment/docker-compose example, but the README does not spell out the exact commands. The docs folder contains PARAMETERS.md for configuration and DEPLOYMENT.md for strategy, which are the places to look for real keys. The project lists supported media server versions, so you must match those to avoid integration issues. For Kubernetes, the separate Helm chart repository (AudioMuse-AI-helm) handles installation. The version 3.3.0 added an experimental -nvidia-arm image for DGX Spark and other GB10 GPU machines, indicating GPU support for acceleration, with details in GPU.md. The plugin system, introduced in 2.6.0, requires a persistent volume mounted on both the Flask and worker containers; otherwise installed plugins are lost on container restart. That is a concrete deployment constraint you need to plan for.
The Features You Get After the Initial Analysis
The README lists several playlist generation modes, all of which require an initial analysis pass over the library. Instant Playlists lets you describe a vibe like high-tempo, low-energy music and get a playlist. Playlist from Similar Songs takes a seed track and finds tracks with a matching sonic signature. Song Paths creates a listening journey between two songs, bridging the sonic gap with intermediate tracks. Sonic Fingerprint generates playlists based on your recent listening habits. Song Alchemy lets you mark tracks as ADD or SUBTRACT to refine a playlist, with a 2D preview and export to the media server. Text Search handles mood, instruments, and genre queries. Lyrics Search goes beyond sound to match themes and stories, but it only works with the 72 languages listed in the README, which is a hard limitation for libraries containing other languages. The Music Map gives a visual 2D genre-based view of the collection. These features are the product's core value: they turn a static library into an explorable space defined by acoustics.
Where It Falls Short: Limitations and Wrong Use Cases
The most obvious limitation is the language restriction for Lyrics Search. If your library contains songs in languages outside the 72 listed, that feature simply will not work for them. The README does not explain how the language detection behaves for mixed-language tracks or instrumental pieces. Another limitation is the operational complexity. AudioMuse-AI is not a single container; it runs Flask and Worker containers, plus PostgreSQL for queueing since version 3.2.0. That is heavier than a typical media server plugin. The plugin system for Navidrome and Jellyfin exists, but the plugin requires a persistent volume, and the Lyrion plugin is unofficial, which means its maintenance is not guaranteed by the core project. The text search and clustering depend on the quality of the audio analysis models, and the README does not disclose accuracy or false positive rates. If you have a small library or you only care about metadata-based organization, the analysis overhead is not worth it. Also, the project lists specific media server versions; if you run an older or newer version, integration may break, and the README does not promise backward compatibility.
The Alternative: Metadata-Driven Smart Playlists
The obvious alternative is to use the built-in smart playlist features of your media server, which rely on metadata and listening statistics. For example, Navidrome has a native smart playlist system that filters on genre, year, play count, and rating. Jellyfin and Plex offer similar tag-based filters. The difference in approach is fundamental: those systems treat music as a set of labels, so a query for high-tempo, low-energy music is impossible unless you have manually tagged every track with tempo and energy. AudioMuse-AI computes those attributes from the audio, so it can answer queries that metadata cannot. However, the metadata approach is simpler, requires no extra containers, and has no analysis delay. Another alternative is a dedicated acoustic analysis tool like Essentia or a CLAP-based embedding script, but those are libraries, not ready-made applications with media server integrations. AudioMuse-AI's advantage is that it packages the whole pipeline into a deployable service with a UI and server connectors, which is exactly what a homelab user needs.
Maintenance, Licensing, and Upgrade Cost
The project is licensed under AGPL-3.0, which has implications if you modify the code and offer it as a network service; you would need to share your modifications. The README does not discuss this, so consult a lawyer if that matters. The repository is actively maintained, with the latest release v3.5.2 from September 2026, and the last push was days later. The release history shows a pattern of regular updates, including maintenance releases and feature additions like text search concept weights in v3.5.1 and a hyperbolic path in v3.5.0. That suggests ongoing development, but it also means you must track releases. The upgrade cost is not documented in the README, but given the multi-container architecture, upgrades likely require pulling new images and possibly migrating the PostgreSQL schema. The queue was moved to PostgreSQL in v3.2.0, so if you are on an older version, you have to change your deployment. The plugin system adds a separate upgrade path, and the unofficial Lyrion plugin may lag behind core releases. Plan for a recurring maintenance task whenever a new version appears.
Editorial conclusion
Adopt AudioMuse-AI if you run a self-hosted music server, want playlists based on acoustics rather than tags, and can handle a multi-container Python deployment. Skip it if you need a single binary, rely on metadata-only search, or require a stable plugin API for every server. Before committing, verify the current Docker Compose example, confirm your media server version matches the listed support (Navidrome 0.62.0, Jellyfin 12.0, LMS v3.69.0, Lyrion 9.0.2, Emby 4.9.1.80, Plex 1.43.2), and check the docs for the plugin mount requirement, since installed plugins vanish on container restart without a persistent volume.
Community notes