Sign Translate turns spoken language into an animated signing avatar, and signing video into speech

Two directions, one intermediate representation

The Key Features section of the README documents two workflows, and the diagrams make the shared idea visible.

The first is Sign Language Production, and it starts from either spoken language audio or spoken language text. The audio path goes through a language identification step and a normalisation step, and the text path goes through normalisation as well. Both converge on SignWriting, which produces a pose sequence, and the pose sequence drives two outputs: a skeleton viewer and a 3D avatar. The audio branch also feeds a Human GAN branch, so the design keeps a person in the loop for the spoken side.

The second is Sign Language Translation, and it runs the other way. You upload sign language video, or use camera sign language video, and the video goes through segmentation and then into SignWriting, which is then turned into spoken language text, and from there into spoken language audio.

The architecture is the interesting part. SignWriting is the pivot in both directions, and that is a meaningful design choice. It is a compact written notation for sign language, so the two halves of the system do not need to talk to each other in pixels or in phonemes. Sign to text goes through video into a written notation into speech, and text to sign goes through the same notation into a pose sequence into an avatar.

Because the notation is shared, improving one direction can improve the other. Because it is shared, an error in the notation stage corrupts both directions at once. The pipeline is elegant and the coupling is real.

The Angular app, the web worker and the service worker

The repository layout is the story of how the work is split.

At the root there is an angular.json, an ionic.config.json, a karma.conf.js, an ngsw-config.json and a .browserslistrc, which is the standard set of files for an Angular application that also targets mobile shells and installs as a progressive web app. The src directory holds the application, and there is a tools directory for the build scripts, a docs directory, and an update.sh and update-dependencies.sh pair at the root.

The tsconfig files tell you more. There are four of them: tsconfig.json, tsconfig.app.json, tsconfig.spec.json and tsconfig.worker.json. The worker configuration is the one that matters for the recognition path, because the model inference and the video segmentation work is what a web worker is for. Running that on the main thread would freeze the interface during processing.

The package.json confirms the shape of the build. The application is an Angular project served with ng serve --host 0.0.0.0, tested with ng test across Chrome and Firefox in headless mode for CI, and built with ng build using a base href that is read from base-path.json at build time, which is how the same build is deployed under a sub-path.

There are also PWA asset scripts that generate the icon set, the favicon, the tile and the splash screens from a source logo, and a separate bundle analyser command that opens the stats output. The deployment script pushes the built dist/sign-translate/browser directory to GitHub Pages, and the CI workflow is also runnable locally through act with the client.yml workflow.

The client runs npm install, npm start and npm test

The Development section is short, which is what you would expect from an application repository rather than a library.

The prerequisite is Node.js, which includes the Node Package Manager.

npm install

The application is then run locally with the start script.

npm start

And the tests run with the test script.

npm test

The scripts reveal a few details worth knowing. The start script has a prestart hook that runs generate:licenses, so the licence inventory for the third party code is regenerated on every run rather than being maintained by hand. The build has a stats variant for the bundle analyser, and the terms and privacy pages are generated from the LICENSE.md and PRIVACY.md files at the root with marked, so the legal text in the application is the same file as the legal text in the repository.

The test scripts are worth a note beyond the obvious. There is a plain test target with code coverage, browser-specific targets for Chrome and Firefox, and a CI target that runs both browsers headless with watch disabled. The package version is 0.0.4, and there are no GitHub releases, so the package is still pre-1.0.

The last push to the repository was on 2026-08-11 and the repository is not archived.

The licence rules out commercial use

The licence situation is the first thing to settle, and it is not a standard software licence.

The badge in the README points at a LICENSE.md file and labels it CC BY-NC-SA 4.0, which is a Creative Commons licence for content rather than an open source licence for software. The licence field recorded for the repository is one GitHub cannot classify, with a pointer to the LICENSE file.

The NC in CC BY-NC-SA is the part that matters commercially. Non-commercial means the project cannot be used to build a paid product, and a communication or accessibility tool for a company is exactly the case that this clause would block. The SA clause requires derivative works to be shared under the same terms, which is workable for a community project and awkward for a company that needs to combine this with code under its own licence.

The application also generates a terms page from the licence and a privacy page from PRIVACY.md, both built by marked during the build. The presence of a privacy document in the repository is notable for a project whose whole point is processing video of people, and it is the document to read if you are considering running this against camera input in a setting where consent matters.

So before any evaluation of the technical quality: read LICENSE.md, and if the intended use is commercial, that is the blocker, not the model.

What the repository does not tell you

There are gaps in the documentation, and for a project at version 0.0.4 they are worth naming rather than glossing over.

The most obvious one is accuracy. There is no number anywhere in the README for how well the recognition works, no word error rate, no benchmark, no comparison against another system. The architecture is documented in detail and the output quality is not quantified. For a translation tool, that is the missing measurement.

The second gap is the vocabulary. The pipeline goes through SignWriting, and SignWriting is a notation that covers a large but finite set of signs. What the shipped models can express, and what they cannot, is not stated in the README. The wiki pages linked from the feature headings, Spoken-to-Signed and Signed-to-Spoken, are where that would live.

The third is the model story. The README mentions models, a Human GAN, and a segmentation stage, and the topic tags include sign-language-recognition, but the repository listing does not show where the model weights live, how they are fetched, or whether they are bundled into the build. The dependency on the @sign-mt configuration package in the prepare hook suggests the model side is assembled from a separate private or public package, which is a reasonable arrangement but an opaque one from the outside.

The fourth is the mobile story. The topic tags list android, ios and pwa, and the ionic.config.json is there, but the README's Development section only covers running the web application. How the native shells are built and what the app store situation is, is not described here.

Where Sign Translate is the wrong tool

The first limitation is the one the licence creates, which I have covered above, and for a commercial assistive product it is disqualifying before any technical consideration.

The second is that this is a web application and a research codebase rather than a library. There is no published API, no importable package, no model you can call from your own backend. The package name in package.json is @sign/translate, and the version is 0.0.4. If you want sign language translation inside an application you are building, you are forking this repository and maintaining the fork, not installing a dependency.

The third is the accuracy gap and the vocabulary gap together. An assistive tool that misrecognises a sign is worse than no tool, because the user has no way to know it was wrong, and the README publishes no way to measure the error rate either.

The fourth is the deployment shape. The application is built as a static progressive web app and deployed to GitHub Pages, and the model work runs in the browser. That is a reasonable choice for a demo and for a privacy-preserving deployment where video never leaves the device, but it also means the compute is the user's device, and the recognition quality on a phone is not the recognition quality on a workstation with a GPU.

The fifth is the framework commitment. Angular, with a specific set of Angular build scripts, a specific test runner and Ionic configuration. The code is well organised, and a team that does not work in Angular will find more friction here than in a framework-neutral implementation would.

How it compares to a hosted translation service

The alternative for a real product is a hosted sign language translation API, and the difference is the one that always comes up with any browser based machine learning tool.

A hosted service puts the models on infrastructure you do not have to run, gives you a latency and accuracy figure in the contract, and can be improved without a redeploy on your side. What it costs you is that the video leaves the device, that the service is a dependency you pay for, and that the vocabulary and language coverage are the vendor's to decide.

Sign Translate inverts all of that. The models run locally in the browser, so the video stays on the device, which matters enormously for a tool that processes continuous camera input of a person using sign language. It also means the improvement cycle is a pull request rather than a vendor roadmap, and that you own the accuracy problem along with everything else.

The intermediate option is to take the models and run them yourself. The architecture here is already shaped for it: the model work is in a web worker, the notation stage is separate from the inference, and the application layer is Angular over a browser. Someone with a server-side inference setup could reuse the pieces and skip the browser entirely. The README does not describe the model interface in enough detail to plan that from the documentation alone, which is why the wiki and the source are the place to look.

The honest summary is that this repository is most valuable as a reference implementation of the two-way pipeline and as a starting point for a self-hosted, privacy-preserving deployment. The hosted route is easier to ship and the accuracy question stays with the vendor.

Editorial conclusion

Sign Translate fits a developer who wants to build on real-time sign language translation, or a team building an assistive interface that needs the transcription or the animation side of it, and who is willing to work in Angular and with a SignWriting intermediate representation. It does not fit a product that needs a signed vocabulary beyond what the shipped models cover, and it does not fit an organisation that needs a compliance-reviewed claim of accuracy, because the repository documents the architecture and the tooling but publishes no accuracy figures. Before adopting, read the wiki pages for Spoken-to-Signed and Signed-to-Spoken, run the app locally with npm start to judge the avatar output for your own use case, and check the licence, which the badge records as CC BY-NC-SA 4.0 and which rules out commercial use.

Frequently asked questions

Official sources

1. Issues
2. Project website
3. README
4. sign/translate on GitHub