TalkingHead renders a lip-syncing avatar in the browser, not on a server

A speaking 3D avatar that runs in the page, not on a server

TalkingHead is a browser JavaScript class that drives a full-body 3D avatar which speaks and lip-syncs in real time. It also maps a set of emoji onto facial expressions, so the face carries something other than mouth movement.

The architectural decision behind it is that everything happens client side. The avatar renders in the visitor's browser and the mouth shapes are computed there, which means the avatar can respond as fast as text or audio arrives rather than waiting for a server to render frames of video. That is what makes conversation possible rather than playback.

The audience is anyone building an interactive character into a web page: a research prototype studying how people talk to embodied agents, a customer-facing assistant with a face, a teaching tool. The project's own list of uses is unusually credible on this point, naming research from several universities, a video conferencing application, and a fully in-browser assistant combining this with local speech recognition, local speech synthesis and a local language model.

The avatar requirements are the real gate

Before anything else, check whether your avatar qualifies, because this is where most attempts stop.

The class supports full-body avatars in the standard 3D interchange format and animations in a widely used motion format. The rig has to be compatible with a specific animation library's skeleton, and the mesh has to carry viseme blend shapes in two named conventions, one from a mobile augmented reality framework and one from a virtual reality platform. Those are not incidental requirements. Blend shapes are the mechanism by which the mouth changes shape, so an avatar without them can be displayed and cannot be made to speak.

The practical consequence is that you do not simply bring any model you like. Either you use avatars from a service that produces compatible rigs, or you prepare your own, which the README documents in an appendix and which the repository supports with a directory of source files for a 3D authoring tool.

This is the sort of constraint that belongs in the first paragraph of an evaluation rather than the tenth, because it decides feasibility before any question of quality arises.

Lip-sync is modular, and that is both the strength and the limit

Speech drives the mouth through lip-sync language modules, and five ship: English, German, French, Finnish and Lithuanian. New languages are added by writing a new module, and the most recent release added two of them from an outside contributor.

That design is the honest answer to a hard problem. Mapping text to mouth shapes depends on the phonology of the language, so a general solution does not exist and per-language work is unavoidable. Making that work a pluggable module rather than a fork is the right structure, and the fact that a contributor added two languages shows the extension point functions.

It is also a hard limit today. If your language is not among the five, you are writing a module, not configuring an option.

Audio can drive the mouth as well as text. A method accepts audio directly, and the most recent release added an option to skip the class's internal break insertion when that audio should be treated as raw. Beyond the default cloud speech service, any external speech synthesis can be used provided it returns word-level timestamps, which the README names explicitly as the integration requirement. That condition is worth checking against your chosen provider, since plenty of speech services return audio and nothing else.

A single dependency, and what the package actually contains

The dependency footprint is one library, and for a project rendering animated 3D that is remarkable.

"dependencies": {
    "three": "^0.180.0"
  },

Everything else is the project's own code, published as a module with its entry point in a modules directory and the package files limited to that directory plus the documentation and licence. A caret range on the rendering library is worth noting, since that library changes its interface more often than most, so an upgrade inside the allowed range is not guaranteed to be uneventful.

The repository itself is larger than the package, because it carries a working test application alongside the class: avatars, animations, poses, audio, example pages, views, a site configuration file and an entry page. That arrangement is generous. You can clone it and see the thing working before deciding whether to adopt the class, which for a visual component is worth more than any amount of documentation.

There is also a directory of authoring-tool source files, which is the practical support for the avatar preparation problem described above.

The demonstrations are framed honestly

One line in the README does more for the project's credibility than the videos it introduces: the demonstration videos are described as real-time screen captures from a browser running the test application, with no post-processing.

For a category where marketing material is routinely rendered offline and presented as if it were live, stating that the capture is real-time and unedited is the claim that matters. It means the frame rate, the lip-sync accuracy and the latency you see are what the class produces.

The listed use cases carry the same quality. They are specific, attributed and mostly linked to papers or working applications rather than described in the abstract, including work presented at academic conferences and a toolkit built by researchers from several institutions. Third-party adoption of that kind is stronger evidence than any self-reported benchmark, because those groups had their own requirements and chose this anyway.

The README also warns about the cost of one integration path, noting that realtime speech-to-speech is considerably more expensive than standard text usage and directing readers to check the provider's pricing. Warning people away from your own most impressive demonstration is not the usual instinct.

Where it will not fit

The avatar requirements come first and have already been covered; they exclude any project without a compatible rig and viseme set, or without the skill to prepare one.

Language coverage is second. Five built-in modules is a real constraint for anything multilingual, and the work to add another is module authoring rather than configuration.

Third, this is a browser 3D application, so it competes for the client's graphics resources. Rendering a rigged character with a physics-driven hair system is not free, and performance on low-end hardware is not something the repository quantifies. The dynamic bones feature and built-in physics described in the release material add to that cost as well as to the quality.

Fourth, the default speech path uses a commercial cloud service, so a working demonstration implies an account and a key unless you substitute a different provider. That substitution is supported and it carries the word-level timestamp requirement.

Finally the cadence. The most recent release is dated 2025-12-08 and the last push 2026-06-02. The project is not moving quickly, which for a stable class with one dependency is less concerning than it would be elsewhere, though the rendering library underneath it does keep moving.

A hosted avatar service is the alternative, and the split is where rendering happens

The alternative is a commercial service that renders a talking avatar as video on its own servers and streams it to you, which the README's own list of related projects mentions as an existing category.

The difference in approach decides almost everything downstream. A hosted service renders on hardware built for it, so visual fidelity can be far higher than a browser will manage, and you supply text and receive video without owning a rig, a blend shape set or a WebGL performance budget. You pay per minute, the round trip adds latency, and the avatar is as interactive as the service's streaming allows.

TalkingHead puts the renderer in the page. After the model loads there is no per-minute cost, response time is bounded by your speech synthesis rather than by video streaming, and the character can react to anything happening in the browser because it is running there. The price is the rigging requirements, the client hardware dependency, and visual quality bounded by what a browser can draw.

For a marketing video, the hosted renderer wins. For a conversation, where the avatar has to answer when the user finishes talking, putting the renderer on the client is the design that makes the interaction feel like one.

Editorial conclusion

TalkingHead suits an interactive character in a web page, where the avatar must answer as soon as the user stops speaking and a per-minute rendering bill would not survive the usage. It is the wrong choice when nobody on the team can supply an avatar with a compatible rig and both viseme blend shape sets, when your language is outside the five built-in lip-sync modules, or when a pre-rendered video would do the job. Settle the avatar question before anything else, and if you intend to use a speech service other than the default, confirm it returns word-level timestamps, because the README names that as the condition for integrating an external one.

Frequently asked questions

Official sources

1. Issues
2. License: MIT
3. met4citizen/TalkingHead on GitHub
4. README
5. Releases