Model or dataset
multimodal-art-projection/YuE avatar
multimodal-art-projection/YuE

YuE2: Symbolic Planning as an Editable Middle Layer in Music Generation

YuE2: frontier music generation with symbolic planning, zero-shot covers, and agentic music editing.

13,483 stars1,558 forksPythonApache-2.0

At a glance

What is it?
YuE2 splits song generation into a melody-and-chord plan and an audio render, which makes the composition inspectable before it becomes sound. The trade-off is a Linux, Python 3.12, 24 GB VRAM environment and a two-stage workflow that assumes you want to read ABC notation.
Who is it for?
Adopt YuE2 if you have an NVIDIA GPU with BF16 support and at least 24 GB of VRAM, you are comfortable in a Python 3.12 virtualenv, and your workflow benefits from a score you can read and edit before rendering. Do not adopt it if you want a single end-to-end call from a text prompt to audio, if you cannot run the model locally, or if your team has no way to review ABC notation.
Can I use it commercially?
Yes. Apache-2.0 is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
Is it still maintained?
Yes. The repository last received commits 14 days ago.
What is it written in?
Mainly Python, according to GitHub's language statistics.

Answers come from the project's GitHub data, last synced on November 14, 2026, and from our analysis. They are not legal advice.

From daily snapshots

Star history

  • multimodal-art-projection/YuE13,483 ★ · +6,619
Star history
Repository2026-09-112026-11-14
multimodal-art-projection/YuE6.9k13.5k

6,619 Stars in the last 64 days, measured from the daily GitHub snapshot. Compare with other projects

Editorial analysis

The problem YuE2 targets: generation you cannot inspect before it becomes audio

Most text-to-music systems hand back a waveform. If the harmony drifts in the second verse, the only remedy is to change the prompt and regenerate, which discards the parts that were already right. YuE2 inserts a representation between the prompt and the render. The README describes the pipeline as writing a melody-and-chord plan first, then realizing that plan as a complete song with vocals and accompaniment. That plan is the product as much as the audio is.

The intended user is not someone who wants a background loop for a video. It is someone who wants to revise a specific bar, or hand a song to a language model agent and ask it to change the key of the bridge. The README's agentic demo follows one track, The Last Train, through 9 steps and 14 versions, moving from Mandarin pop to English jazz with new harmony and a saxophone solo. That workflow only makes sense if intermediate state is addressable, which is what the symbolic plan provides.

The secondary audience is cover work. A transcribed recording becomes a melody score, new lyrics or a new style prompt are applied, and the accompaniment is regenerated around the retained melody.

One backbone, two prediction modes, and a VAE at the end

The architecture is described as an AR-NAR Mixture-of-Transformers backbone. The autoregressive half predicts the score and semantic music tokens. The non-autoregressive half, using flow matching, generates acoustic latents. A VAE decodes those latents into stereo audio at 48 kHz without quantization, per the quick start notes.

What differs between creating, covering, and editing is not the model but the origin of the score. In creation, YuE2 writes it. In covering, it comes from a transcribed recording via SheetSage2. In editing, it comes from a composition a person or agent has already modified. That is a clean separation: one checkpoint serves all three, and the README states this explicitly.

The Python surface mirrors the stages: plan(), generate_semantic(), synthesize(), decode(). The generation guide is cited for exact-plan reuse and decoder selection, which implies you can re-render from a stored plan without re-planning.

Installing and running the first song

The stated environment is Linux, Python 3.12, and an NVIDIA GPU with BF16 support and 24 GB of VRAM. Model files download from Hugging Face on first use. The repository is a Python package, yue2-infer 0.1.6 in pyproject.toml, with a yue2 console script, and examples/ holds generate.py, song.json and sample scores such as score.abc and melody.abc.

The README also links the yue2-v0.1.6 release, and points to experimental low-memory, Windows, AMD, Apple Silicon and Intel paths.

Where the approach gets in the way

The 24 GB VRAM NVIDIA setup is the main documented path; the README sends low-memory, Windows, AMD, Apple Silicon and Intel users to experimental paths.

The symbolic layer is also a commitment. Scores are ABC notation, as the .abc files in examples/ show, and covers start from a SheetSage2 transcription, so someone has to read and check that notation. A team without anyone who reads a lead sheet loses the project's distinguishing feature.

Covers also depend on a second model, SheetSage2, published separately on Hugging Face.

Finally, the headline quality number needs reading carefully. The 6.9632 SongBench Avg is described as YuE2 best-of-8, the highest observed mean among all evaluated settings on 192 WildSongBench prompts. Best-of-8 means eight candidates were generated and the best selected. That is a legitimate way to report a ceiling, but it is not the score you get from a single call.

How this differs from end-to-end audio generation

The obvious comparison is a text-to-audio model that maps a prompt straight to a waveform with no intermediate score. The difference is not quality, it is where correction happens. In an end-to-end system, correction means editing the prompt and accepting a wholly new render. In YuE2, correction can mean editing the plan and re-rendering, which is why the README frames melody and chords as explicit controls that a person or an agent can inspect and edit.

That control has a cost. You carry an extra representation, you need tooling to read it, and you inherit whatever errors the planner makes. An end-to-end model has no planner to be wrong. For a one-off jingle where nobody will revise the harmony, the plan is overhead.

The interesting middle ground is the editing loop. Because the score, style, and lyrics are separate inputs, an agent can revise one without regenerating the others. The README's 14-version demo is the case study for this. Whether that loop is faster than prompt iteration depends on how often your revisions are musical rather than atmospheric.

Licence, releases, and what upkeep looks like

The repository is not archived. GitHub metadata and pyproject.toml list Apache-2.0, but the tree also ships a separate MODEL_LICENSE file alongside THIRD_PARTY_NOTICES.md and a licenses/ directory, all listed as license files in pyproject.toml. The model files are hosted separately on Hugging Face under m-a-p/YuE2-3B. If you plan to ship generated audio commercially, read LICENSE and MODEL_LICENSE before relying on either. This is not legal advice.

The project also carries a version boundary worth noting. The README opens by pointing to a YuE-v1 branch, where the original code, documentation, and license are preserved. The current main branch is a different system with a different API, so any tutorial or script written against YuE v1 will not transfer. The most recent model release is yue2-v0.1.6, dated 2026-09-09, followed by the yue2-music-v1.2.0 agent skill release on 2026-09-25.

Upgrade cost is mostly environmental. Model files are fetched on first use rather than vendored, so a version bump means re-downloading weights and re-checking that your GPU still meets the BF16 and 24 GB requirement.

Editorial conclusion

Adopt YuE2 if you have an NVIDIA GPU with BF16 support and at least 24 GB of VRAM, you are comfortable in a Python 3.12 virtualenv, and your workflow benefits from a score you can read and edit before rendering. Do not adopt it if you want a single end-to-end call from a text prompt to audio, if you cannot run the model locally, or if your team has no way to review ABC notation. Before committing, verify three things against the repository: that the 3B checkpoint downloads and fits your card, that the yue2-v0.1.6 release matches your Python version, and that the benchmark protocol in docs/benchmarks.md is one you accept, since the headline 6.9632 SongBench Avg is a best-of-8 figure rather than a single-sample result.

Frequently asked questions

Is YuE2 the best local AI for music generation?

YuE2 runs on your own hardware, but its quick start asks for Linux, Python 3.12 and an NVIDIA GPU with BF16 support and 24 GB VRAM. Its own evidence is a 6.9632 SongBench Avg at best-of-8 on WildSongBench, which the README calls competitive with the evaluated proprietary systems.

Which open source model is considered the best for generating music, and is YuE one?

The repository metadata lists Apache-2.0, but the tree also ships a separate MODEL_LICENSE for the model, so the licence terms need checking before calling it open source. The README reports YuE2 as competitive with the evaluated proprietary systems on WildSongBench.

What AI music generator is available on GitHub, and how do you get YuE2?

YuE2 is cloned from https://github.com/multimodal-art-projection/YuE.git (repository metadata says Apache-2.0, and a separate MODEL_LICENSE ships alongside LICENSE), with weights pulled from m-a-p/YuE2-3B on Hugging Face on first use. The original YuE's code, documentation and license are preserved on the YuE-v1 branch.

Official sources

  1. License: Apache-2.0
  2. multimodal-art-projection/YuE on GitHub
  3. Project website
  4. README
  5. Releases
Community

Where developers are discussing it

Posts on Hacker News, dev.to and Lobsters that link to this repository, found by our scan of those communities. 0 in the last 30 days; newest first.

Add this badge to your README

If you maintain this project, the badge below links readers to this analysis and shows its maintenance status from the daily GitHub snapshot. Paste the markdown into your README; add ?metric=license or ?metric=stars to the image URL for a different field.

Add this badge to your README

markdown
[![Hysen Labs](https://hysenlabs.com/badge/multimodal-art-projection-yue.svg)](https://hysenlabs.com/projects/multimodal-art-projection-yue)

Star history chart for your README

markdown
[![Star history](https://hysenlabs.com/chart/multimodal-art-projection-yue.svg)](https://hysenlabs.com/projects/multimodal-art-projection-yue)

Was this useful?