GAGAvatar: animatable Gaussian head avatars from a single image

One-shot reconstruction and real-time reenactment from a single photo

GAGAvatar reconstructs controllable 3D head avatars from single images, and the README states that it achieves one-shot 3DGS-based head reconstruction and real-time reenactment. The project is a NeurIPS 2024 paper by Xuangeng Chu and Tatsuya Harada of The University of Tokyo and RIKEN AIP, with an arXiv entry at 2410.07971 and a project page.

The two halves of that sentence describe the two halves of the system. Reconstruction is one-shot: a single photograph is turned into a 3D representation, with no multi-view capture session and no subject cooperation beyond the photo. Reenactment is the driving side: the reconstructed avatar is animated in real time from a driving signal, which the README shows can be another image, a tracked video, or a tracked image database.

The representation is 3D Gaussian Splatting, and the README is explicit that the bundled renderer is not the stock one. The number of channels was changed so that 3D Gaussians carry 32-dim features, and the package was renamed to avoid a conflict with the original Gaussian splatting implementation. The fork lives at the separate `xg-chu/diff-gaussian-rasterization` repository, and the 32-dimensional feature vector is what makes the avatar controllable rather than merely a fixed appearance.

The driving data comes from a companion project, GAGAvatar_track, which is a separate repository with its own data tracker and its own resource setup step. The acknowledgements also credit work built on FLAME, StyleMatte, EMICA and VGGHead, which tells you the head model, the matting, the expression and pose components and the face detector all come from elsewhere and are dependencies of this project rather than parts of it.

The install path is a conda environment and a custom CUDA rasterizer

The installation is four steps, and two of them involve compiling code.

The clone has to recurse into submodules, since the repository has a `.gitmodules` file at the top level:

git clone --recurse-submodules [email protected]:xg-chu/GAGAvatar.git
cd GAGAvatar

Then the environment, which is a conda environment file rather than a requirements list:

conda env create -f environment.yml
conda activate GAGAvatar

Third, the 3DGS renderer, which is a separate repository built and installed as a local package and then removed:

git clone --recurse-submodules [email protected]:xg-chu/diff-gaussian-rasterization.git
pip install ./diff-gaussian-rasterization
rm -rf ./diff-gaussian-rasterization

The `rm -rf` at the end is not a mistake, it is cleanup of the clone after the local install.

The fourth step is assets, and there are two ways to do it. The script is the original route:

bash ./build_resources.sh

And the README describes the newer packaged path, which runs when GAGAvatar is installed as a package:

gagavatar-assets download --root assets

That command downloads only assets with explicit upstream sources and verifies known sizes and SHA-256 hashes, which is a meaningfully better supply chain story than a script that fetches whatever it finds. The README is equally clear about what it does not do. FLAME and tracker resources are represented in the manifest but are not downloaded automatically, because they have separate setup or license requirements.

The tracker resources are prepared separately, from inside the tracking submodule:

cd core/libs/GAGAvatar_track
bash ./build_resources.sh

The Python side of the install is light by comparison. The `pyproject.toml` declares a setuptools build backend requiring `setuptools>=68`, a package named `gagavatar` at version 0.1.0, `requires-python` of 3.11 or newer, and a single console script, `gagavatar-assets`, pointing at `gagavatar.assets:main`. The package data includes an `assets_manifest.json`.

So the honest summary of the install is that conda holds the heavy scientific dependencies, a CUDA rasterizer has to compile against your GPU toolchain, and the assets arrive through a hash-verified downloader with two documented exceptions.

Three driving modes, and the image path is the slow one

The Quick Start Guide gives three invocation shapes, and the differences between them are the differences between the driving data sources.

Driven by another image:

# This will track the images online, which is slow.
python inference.py -d ./demos/examples/2.jpg -i ./demos/examples/1.jpg

Driven by a tracked video:

python inference.py -d ./demos/drivers/obama -i ./demos/examples/1.jpg

Driven by a tracked image database:

python inference.py -d ./demos/drivers/vfhq_demo -i ./demos/examples/1.jpg

The `-i` argument is the source image being reconstructed and the `-d` argument is the driving input. The comment on the first example is the one to notice. Tracking an image online is slow, because the tracker has to run over the image to work out the expression and pose. A tracked video or a tracked database skips that, because the tracking result is already computed and stored.

That distinction also explains the training pipeline, which is built around the same idea. The tracker produces an `optim.pkl` per frame, and the training set is assembled by pairing that tracking data with the images in an LMDB.

For anyone wanting to work with their own material, the README points to specific places rather than giving a general guide. To test the inference speed, look at the `speed_test()` function in `inference.py`. To test your own images online, refer to lines 52-55 in the same file. To test your own driving sequences, the README points at the GAGAvatar_track repository and the demo sequences for building your own driving sequence.

Retraining is four steps over an LMDB and a tracking pickle

The Training Guide is inside a collapsed section and starts by saying the pre-trained model can be used directly, with retraining needed only for your own data. The pipeline is documented concretely enough to follow.

Step one is building the image LMDB, and the README is emphatic that you build it yourself. All the images should be cropped as inference does, with a pointer to line 218 in `core/libs/GAGAvatar_track/engines/engine_core.py`. The images are dumped with `core/libs/utils_lmdb.py`, which also offers an API for building the LMDB, a `dump(key_name, payload)` function, where the payload should be a tensor with shape (3, 512, 512) in the range [0, 255]. The key format is a video id and a frame id, and the README explains that 015252 is the video id used when sampling while 99 is the frame id, where 0 is the first frame and other frame ids can be discontinuous:

img_lmdb:
    '015252_99' : image payload

Step two is tracking the LMDB with `track_lmdb.py` in GAGAvatar_track to produce an `optim.pkl`, whose structure is a dictionary keyed by frame with the tracking values per frame:

optim.pkl:
- dict_keys(['000000_0', …])
    - "000000_0": dict_keys(['bbox', 'shapecode', 'expcode', 'posecode', 'eyecode', 'transform_matrix'])

The per-frame keys are the readable form of what the tracker produces: a bounding box, shape code, expression code, pose code, eye code and a transform matrix.

Step three is building `dataset.json` yourself, and it should contain the keys present in `img_lmdb` and `optim.pkl`:

dataset.json: {
    "train": ["000000_0", "000000_5", ..., '001384_654'],
    "val":   ["015209_0", ..., "015218_7"], 
    "test":  ["015203_0", ..., "015252_139"]
}

Step four is changing the config and training, and the command names the config and the dataset:

python train.py --config gaga --dataset vfhq

The `configs/` directory at the repository root is where those config files live, and the `--dataset vfhq` argument points at a dataset name, which corresponds to the `vfhq_demo` driver directory used in the quick start.

A research release with no published version history

The maintenance picture needs stating plainly, because the packaging metadata and the release history tell different stories.

The repository is not archived and the last push was on 2026-09-16, so the code is still being worked on. The recent releases list is empty, though, in the sense that no GitHub releases are attached to the repository. And the version in `pyproject.toml` is 0.1.0, which is a research code drop rather than a maintained package line.

That combination is normal for this kind of release and it is what you should plan for. A NeurIPS paper's repository is published so that results can be reproduced and so that others can build on the method, not so that it can be depended on. There is no changelog discipline, no semantic versioning of a public API and no stability promise for the `gagavatar` package.

What has been maintained is the research code itself. The README shows a project page with more results, a YouTube video, an arXiv PDF and a separate data tracker repository, and the code is arranged to support the paper's experiments, with a `demos/` directory holding example images, a driver video and a demo image database, and a `gagavatar/` package directory alongside `inference.py` and `train.py` at the root.

For adoption, the practical rule is to treat this as a codebase to read and fork rather than a dependency to install. If you need the method, read the code, cite the paper, and accept that you own the maintenance of whatever you build on it.

Compared with a packaged avatar generator and with a tracked-video-only pipeline

Two alternative approaches cover most of the same ground, and the difference in approach is about what you supply and what you get.

A packaged avatar generator, of the kind that produces a rigged or expression-blended face from a photo, is a service or a product with a documented interface. The difference is that those systems hide the 3D representation entirely and give you a face that can be driven, whereas GAGAvatar produces a 3D Gaussian representation with 32-dimensional per-Gaussian features, which is what makes the result controllable in a way that a blended 2D face is not. The cost of that control is that you now own the whole pipeline, including the rasterizer.

The second alternative is a tracking-only setup, where you track a driving video with a face tracker and blend it onto a single image. This is the classic face reenactment approach, and it is much lighter, because it needs no 3D reconstruction and no custom CUDA kernel. The trade-off is that the output is limited to what the driving video contains, and a single driving sequence cannot produce a genuinely novel pose or a novel lighting condition, whereas a reconstructed 3D avatar can.

GAGAvatar sits between them. It does the reconstruction step, and it accepts a driving input that can be as cheap as a single image, at the cost of running the tracker yourself when your driving input is a raw image.

One more comparison is worth naming for anyone working with faces. The acknowledgements list FLAME, StyleMatte, EMICA and VGGHead as upstream components, and the asset manifest deliberately does not download FLAME and the tracker resources automatically because of their separate setup and license requirements. That is a sensible choice, and it also means an adopter has to clear those licenses independently, which is a real step and not a formality.

Editorial conclusion

Adopt GAGAvatar if you are doing research on animatable head avatars and want the reference implementation and training pipeline rather than a packaged tool. Do not adopt it if you need a stable library with a published release history, since the repository has no GitHub releases and the package version in pyproject.toml is 0.1.0. Verify first that your GPU and driver support the custom rasterizer build, and read the resource licensing notes, because FLAME and the tracker resources are deliberately not downloaded by the asset tool because they carry separate setup and license requirements.

Frequently asked questions

Official sources

1. Issues
2. License: MIT
3. README
4. xg-chu/GAGAvatar on GitHub