whisper_android: Offline Whisper Speech to Text on Android with TFLite

What whisper_android Solves, and for Whom

The repository targets a narrow problem: running OpenAI Whisper speech recognition on an Android device with no network round trip. The README states the project offers two Android apps, one using the TensorFlow Lite Java API and one using the TensorFlow Lite Native API, plus a Python script for model generation and pre-built APKs. The audience is Android developers who want to embed transcription rather than call a cloud endpoint, and who are willing to work at the level of model files, buffers and listeners.

That framing matters because the project is not a library with a versioned artifact. It is a set of applications you read, copy from and adapt. The whisper_java app is the easier entry point for Java developers, while whisper_native is aimed at developers who prefer native code and want the performance path. If you want a drop-in dependency with a semantic version, this is the wrong shape of project. If you want a working reference implementation of Whisper on TFLite that you can fork and reshape, the folder layout gives you exactly that.

Inside whisper_java and whisper_native: Two Inference Paths

The top level of the repository splits the Android work into whisper_java and whisper_native, with models_and_scripts holding generate_model.py and a generated_model directory of optimized TFLite models, and demo_and_apk holding pre-built APKs. The split is not cosmetic. The Java app drives inference through the TensorFlow Lite Java API, which keeps everything in Java and is straightforward to wire into an existing Activity or Service. The native app drives the same class of model through the TensorFlow Lite Native API, which the README describes as offering optimized performance for developers preferring native code.

The integration surface exposed in the guide is small and listener-based. A Whisper instance is constructed with a Context, then loadModel takes a model path, a vocabulary path and a boolean for multilingual mode. Results and status updates arrive through IWhisperListener, with onUpdateReceived for status strings and onResultReceived for transcribed text. Audio enters either as a file path via setFilePath or as float samples pushed through writeBuffer, which is how the Recorder example suggests feeding live audio. Actions are selected with setAction, and the documented constant is Whisper.ACTION_TRANSCRIBE. Start and stop are explicit calls, so the caller owns the lifecycle.

The audio contract is stated plainly: input should be 16K, mono, 16bits, and the Recorder class is documented as recording in that same format. Any mismatch between your capture pipeline and that contract is your bug, not the library's. The README's own important note concedes that handling audio data and transcriptions may require careful synchronization and error handling, which is a fair description of where the real work sits.

Installing whisper_android and Running a First Transcription

There is no Gradle coordinate to add. The README's usage section says to navigate to the whisper_java folder, open the project in Android Studio, and build and run on an Android device or emulator. The same steps apply to whisper_native. So the install path is a clone and an Android Studio build, not a package manager command.

If you only want to see the thing work, demo_and_apk contains pre-built APKs for direct Android installation. That is the fastest way to judge whether the on-device quality is acceptable on your hardware before you invest in integration.

For integration, the guide's initialization pattern is the starting point. Note that the model and vocabulary paths are placeholders you must replace, and the third argument toggles multilingual mode:

Whisper mWhisper = new Whisper(this);
String modelPath = "path/to/whisper-tiny.tflite";
String vocabPath = "path/to/filters_vocab_multilingual.bin";
mWhisper.loadModel(modelPath, vocabPath, true);
mWhisper.setListener(new IWhisperListener() {
    @Override
    public void onUpdateReceived(String message) { }
    @Override
    public void onResultReceived(String result) { }
});

Transcription of an existing file is three calls. The README repeats the 16K, mono, 16bits requirement next to setFilePath, so convert your audio before pointing at it:

String waveFilePath = "path/to/your_audio_file.wav";
mWhisper.setFilePath(waveFilePath);
mWhisper.setAction(Whisper.ACTION_TRANSCRIBE);
mWhisper.start();

For live audio, the Recorder class records in the required format and hands you float samples through onDataReceived. The README's own comment shows the intended bridge into Whisper, so the loop is capture, forward, transcribe:

Recorder mRecorder = new Recorder(this);
mRecorder.setListener(new IRecorderListener() {
    @Override
    public void onUpdateReceived(String message) { }
    @Override
    public void onDataReceived(float[] samples) {
        mWhisper.writeBuffer(samples);
    }
});

Before recording, the guide calls checkRecordPermission(), and it closes with a note to handle permissions, error handling and file path management yourself. Expect to write that code.

Where whisper_android Breaks Down

The first limitation is the model pipeline. The repository ships generate_model.py and a generated_model directory, which means model conversion is part of your build, not a fixed input. The README does not document which Whisper checkpoints the script supports, what the conversion costs in time or memory, or how to validate that a converted model behaves like the original. If you change the checkpoint, you are on your own to confirm the output is still correct.

The second is the absence of releases. No releases were retrieved for this repository, so there is no changelog to read before upgrading and no tagged artifact to pin. Upgrading means diffing the master branch yourself. The last push was on 2026-03-18, which is recent enough that the code is not stale, but a commit history is not a compatibility contract.

The third is the API's silence on the hard parts. The README documents loadModel, setFilePath, setAction, start, stop, writeBuffer and the two listeners. It does not document threading rules for writeBuffer, what happens when start is called twice, how to cancel mid-transcription, or how memory is reclaimed after stop. Its own important note points at synchronization and error handling as open concerns. For a long-running dictation feature, those are the questions that decide whether the integration survives contact with users.

Finally, this is the wrong tool if you need high accuracy on varied accents and noisy rooms with a small model, or if your app already depends on a platform speech API that is good enough. On-device Whisper trades accuracy and latency for privacy and offline operation, and that trade is not always the right one.

whisper_android Compared with a Cloud Speech API

The obvious alternative is a hosted speech-to-text service from a major cloud provider. The difference is architectural, not cosmetic. A cloud API sends audio off the device, returns text, and keeps the model current without you shipping anything. whisper_android runs inference locally through TensorFlow Lite, so audio never leaves the phone, transcription works with the radio off, and there is no per-minute cost. In exchange, you own the model file, the binary size it adds, the conversion script, the thermal and battery cost of local inference, and the accuracy ceiling of whichever Whisper checkpoint you converted.

Within the repository itself, the Java and native apps are also alternatives to each other. Choosing whisper_java buys simpler integration and a smaller amount of native build surface. Choosing whisper_native buys the performance path the README describes, at the cost of working in native code. That is the real decision most readers face, and it should be made by measuring on the target device rather than by reading the folder names.

The project also credits Niranjan Yadla's whisper.tflite as the original TFLite implementation of OpenAI Whisper for on-device automatic speech recognition. If you want the upstream port rather than an Android application built around it, that repository is the place to look.

Maintenance, Licensing and Upgrade Cost

The repository is not archived, and the last push was on 2026-03-18, so the code has moved within the last six months. That is the whole of what the available facts support about maintenance. There are no retrieved releases, so there is no version cadence to plan around and no upgrade notes to consult. The README lists a PayPal link and an email address for inquiries and business discussions, which suggests a single-maintainer project rather than a team with a support rotation. Budget for reading the source when something changes.

The licence is MIT, which is permissive and places few obligations on how you redistribute the code. Two caveats are worth stating without giving legal advice. First, the licence file is LICENSE.txt at the repository root; read it rather than relying on the GitHub label. Second, the README separately acknowledges Niranjan Yadla's whisper.tflite as the original TFLite implementation, and the Whisper models themselves originate from OpenAI. Model weights and third-party ports can carry their own terms, so check those independently before shipping a commercial product. The repository also includes privacy_policy.md, which is relevant if you publish an app built on this code.

Editorial conclusion

Adopt whisper_android if you need on-device transcription and are prepared to own audio capture, model conversion and threading yourself. Do not adopt it if you expect a maintained SDK with releases, a documented API surface or a support channel beyond the maintainer's email. Before committing, build whisper_native on your target device, run generate_model.py on the exact Whisper checkpoint you intend to ship, and confirm the 16 kHz mono 16-bit input contract holds in your recorder path.

Frequently asked questions

Official sources

1. Issues
2. License: MIT
3. README
4. vilassn/whisper_android on GitHub