TensorFlowASR: End-to-End Speech Recognition in TensorFlow 2, and What It Costs
:zap: TensorFlowASR: Almost State-of-the-art Automatic Speech Recognition in Tensorflow 2. Supported languages that can use characters or subwords
At a glance
- What is it?
- TensorFlowASR implements DeepSpeech2, Jasper, RNN Transducer, ContextNet and Conformer with TFLite export. It is a research toolkit that expects you to clone it, not pip install it.
- Who is it for?
- Adopt TensorFlowASR if you already train in TensorFlow 2 and want Conformer or ContextNet transducer code with a TFLite export path; skip it if you need a pip-installable inference library or a maintained serving stack.
- Can I use it commercially?
- Yes. Apache-2.0 is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository last received commits 6 days ago.
- What is it written in?
- Mainly Python, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
What TensorFlowASR solves, and who it is actually for
Most speech recognition code you can install with a single pip command is a wrapper around someone else's trained weights. TensorFlowASR is the opposite kind of project. It implements the architectures themselves in TensorFlow 2: DeepSpeech2 and Jasper as CTCModel variants trained with CTC loss, and Conformer, ContextNet, Streaming Transducer and RNN Transducer as transducer models trained with RNNT loss. The README describes the result as "Almost State-of-the-art Automatic Speech Recognition in Tensorflow 2" and lists supported languages as those that can use characters or subwords, which is a way of saying the text side is configurable rather than fixed to English graphemes.
The intended user is someone who has audio and transcripts and wants to train an end-to-end recognizer in TensorFlow rather than PyTorch. That is a narrow but real audience. If you are choosing an ASR stack today, the practical question is not which model is best but which framework your team can debug at 2am. TensorFlowASR exists for teams that already answered TensorFlow. It is not a hosted API, it does not ship a command that turns an arbitrary podcast into text, and the README points you at example folders rather than a product. The repository is not archived and the last push was on 2026-09-13, so the code is current, but currency is not the same as support. There is one maintainer listed, Huy Le Nguyen, with a single contact email.
Two training paths: CTC models and transducer models
The architecture split matters more than the model names. CTCModel covers DeepSpeech2 and Jasper and trains against CTC loss, which assumes the output frames are conditionally independent given the input. Transducer models (Conformer, ContextNet, Streaming Transducer, RNN Transducer) train against RNNT loss, which keeps an internal prediction network and therefore models the text history. The README's own framing is "End2end models using RNNT Loss for training" versus "End2end models using CTC Loss for training", and it links each architecture to a paper and an examples directory, for example examples/models/transducer/conformer and examples/models/ctc/deepspeech2.
That choice propagates into everything downstream. RNNT loss is not a stock TensorFlow op, which is why the installation section says you should use git clone "for installing necessary packages from other authors (ctc_decoders, rnnt_loss, etc.)". Those are compiled or externally sourced pieces, and they are the reason a plain pip install of the package does not give you a working training loop. The repository reflects this in its layout: setup.sh, a Dockerfile, docker-compose.yml and separate requirement files for apple, cpu, gpu, tpu, dev and text. Feature extraction and augmentation are documented in their own READMEs under tensorflow_asr/features and tensorflow_asr/augmentations rather than in the top-level file. If you only read the front page, you will miss both.
Installing TensorFlowASR from source with setup.sh
The README is explicit that training and testing should start from a clone, so the first step is fetching the repository. The note for Apple Silicon is that TensorFlowASR requires python >= 3.12 there, which is stricter than the python >= 3.8 shown in the badge for the project generally.
git clone https://github.com/TensorSpeech/TensorFlowASR.git
cd TensorFlowASR
./setup.sh [apple|tpu|gpu] [dev]The first argument selects the hardware target and the second is optional and installs development dependencies. The README also says to see the requirements.[extra].txt files for extra dependencies, and the repository contains requirements.apple.txt, requirements.cpu.txt, requirements.gpu.txt, requirements.tpu.txt, requirements.dev.txt and requirements.text.txt. Those files are read by setup.py to build extras_require, so the same sets are reachable through the package metadata.
If you would rather not touch your host Python, the README offers a container path, and docker-compose.yml builds the image with using_gpu set to "true" and install_rnnt_loss set to "true", runs with the nvidia runtime, and publishes port 6006, which is TensorBoard's default port.
docker-compose up -dAfter that, the README does not walk you through a first transcription. It redirects: training to docs/tutorials/training.md, testing to docs/tutorials/testing.md, TFLite conversion to docs/tutorials/tflite.md, and pretrained results to per-example result folders such as examples/models/transducer/conformer/results/sentencepiece/README.md. Follow those files rather than guessing at entry points. One detail worth knowing before you write a data pipeline: the README states that Keras builtin training uses an infinite dataset, which avoids the potential last partial batch.
TFLite export is the deployment story, and it is a real constraint
The most distinctive claim in the README is that after conversion the TFLite model behaves like a function mapping an audio signal directly to text and tokens. That is an appealing shape for mobile and edge deployment, and it is the reason the repository carries topics like tflite-convertion and tflite-model. The mechanics are documented in docs/tutorials/tflite.md, not on the front page.
Treat this as a boundary rather than a bonus. TFLite conversion constrains which operations your model can use, and transducer models with RNNT loss are the harder case because the loss itself is an external dependency at training time. The README does not document rollback, does not describe what happens when a conversion fails, and does not list which of the six architectures convert cleanly. That silence is the thing to test first on your own audio, because discovering it after training a Conformer for days is expensive. The same caution applies to streaming: Streaming Conformer and Streaming Transducer are listed as supported architectures with paper references, but the README does not specify latency figures or chunk sizes, so any real-time claim has to come from your own measurements.
Where TensorFlowASR is the wrong tool
If you want speech-to-text output and not a training framework, this is the wrong dependency. There is no documented single command that transcribes a file, and the README's pretrained section is a pointer to example result folders rather than a model hub with a download API. You will be assembling feature extraction, a checkpoint, a tokenizer and a decoder yourself.
The second mismatch is framework. Everything here is TensorFlow 2, and requirements.txt pins keras-nightly and a version of ipython below 9.0.0, which tells you the dependency surface moves. If your team writes PyTorch, porting an architecture from this repository is more work than starting from a PyTorch toolkit, and the papers linked in the README are framework-neutral, so nothing is lost by reading them directly. The third mismatch is scale. The corpus table lists LibriSpeech at 970h and Common Voice at 1932h for English, and Vivos at 15h, InfoRe Technology 1 at 25h, InfoRe Technology 2 at 415h and VietBud500 at 500h for Vietnamese. If your labelled audio is a few hours, you are in fine-tuning territory, and the README's example results are the reference point you should compare against rather than expect to beat.
How it differs from ESPNet and NeMo in approach
The README credits ESPNet, described as an end-to-end speech processing toolkit in PyTorch, alongside NVIDIA OpenSeq2Seq, warp-transducer and a ContextNet implementation. ESPNet is the closest honest comparison: it covers speech recognition and other speech tasks in PyTorch, with its own recipe system, while TensorFlowASR is TensorFlow-only and narrower in task scope. The difference in approach is not model quality, since both implement Conformer and related architectures from the same papers. It is the surrounding machinery. ESPNet organizes work as per-corpus recipes; TensorFlowASR organizes it as examples/models/<loss-type>/<architecture> with separate documentation for features, augmentations and TFLite conversion. If your deployment target is TensorFlow Lite, that conversion path is what TensorFlowASR is built around, and it is the reason to accept the smaller ecosystem. If your target is a server running PyTorch, ESPNet's breadth is the better trade. NVIDIA OpenSeq2Seq is credited as an influence but is not a drop-in substitute for this codebase's structure.
Licence, dependencies and the cost of staying current
TensorFlowASR is Apache-2.0, and setup.py carries the standard Apache header with the copyright held by Huy Le Nguyen. Apache-2.0 is permissive and includes an explicit patent grant, which matters if you ship a converted model inside a product. That is a description of the licence text, not legal advice; if you are redistributing a trained checkpoint or a TFLite file, have your own counsel review the notices you must carry.
The upgrade cost is the part people underestimate. The project reached v3.0.0 on 2025-06-11, after v2.1.0 on 2024-06-09 and v2.0.1 on 2024-05-19, so major versions are roughly annual and the jump from 2.x to 3.x is not a patch. setup.py declares version 3.0.0 and classifiers that still list Python 3.6 and 3.7, while requirements.txt pins keras-nightly, which is a moving target by design. Expect to re-resolve requirements on every TensorFlow or Keras bump. The repository does include tests/ with pytest configured in pyproject.toml (testpaths = "tests", filterwarnings turning warnings into errors), which gives you a way to check an upgrade rather than guessing. Run that suite on a branch before you move a training run onto a new TensorFlow release.
Editorial conclusion
Adopt TensorFlowASR if you already train in TensorFlow 2 and want Conformer or ContextNet transducer code with a TFLite export path; skip it if you need a pip-installable inference library or a maintained serving stack. Before committing, check the setup.sh branch you need (apple, tpu or gpu), open examples/models/transducer/conformer/results/sentencepiece/README.md to see what the published checkpoints actually score, and confirm the requirements.cpu.txt, requirements.gpu.txt and requirements.tpu.txt sets resolve on your Python version.
Frequently asked questions
What is TensorFlowASR used for?
It is used to train and test end-to-end automatic speech recognition models in TensorFlow 2, including DeepSpeech2, Jasper, Conformer, ContextNet, RNN Transducer and Streaming Transducer, and to convert those models to TFLite for deployment.
How does TensorFlowASR compare with PyTorch speech recognition toolkits?
The README credits ESPNet, an end-to-end speech processing toolkit in PyTorch, among its references, and both implement architectures from the same papers. The practical difference is the framework and the deployment path: TensorFlowASR is TensorFlow-only and built around TFLite conversion.
Is TensorFlowASR a framework or a library?
It is distributed as a Python package named TensorFlowASR with setup.py and a pyproject.toml, but the README says training and testing should use git clone so that packages from other authors such as ctc_decoders and rnnt_loss are installed. Treat it as a source checkout you build on rather than a drop-in library.
Community notes