Whispering Tiger turns any audio stream into live subtitles for VRChat and OBS

It listens to a stream and writes to a browser

Whispering Tiger is a free and open source tool that can listen or watch to any audio stream or in-game image on your machine, and prints out the transcription or the translation to a web browser using WebSockets, or over OSC.

The examples the documentation gives are streaming overlays and VRChat, and there is also a Desktop mode for live streaming applications such as OBS, vMix and XSplit. The two output mechanisms are the important part of the design.

A WebSocket output means a browser page receives the text, which is what an overlay does. An OSC output means a virtual tabletop or a streaming tool receives the text over a protocol it already speaks. VRChat is the case the project cares about most, and the repository shows that with a VRC_OSCLib.py and a VRC_OSCServer.py sitting at the top level, plus a websocket.py and a websocket_clients directory.

The privacy property is the one the documentation leads with. It runs 100 percent locally on your machine, and once the AI models are downloaded no further internet connection is required. For a streamer or a VRChat user, that is the difference between a captioning tool and a microphone upload.

Six speech models, and they are not interchangeable

The speech recognition section lists the models the tool can drive, and the differences between them are the practical decision.

OpenAI's Whisper supports about 98 languages. Meta's multimodal translation and speech model supports about 101 and is not audio only, so it also handles the image side. Microsoft's Speech T5 is English only. Microsoft's Phi-4 Multimodal LLM supports about 23 languages. NVIDIA's NeMo Canary covers English, Spanish, German and French, and Wav2Vec Bert 2.0 covers English and German.

So the choice is between a wide language coverage with a text-oriented model, a smaller set with a dedicated speech model per language, and a large multimodal model that can also see.

The model choice interacts with the hardware. The documentation notes that CUDA for GPU acceleration is recommended, and the release notes mention that after downloading a medium Whisper model and a medium NLLB-200 translation model it can take up to 20 GB, which tells you these are not small downloads.

The dependencies reflect the same reality. The requirements file pins transformers, ctranslate2 and faster-whisper style tooling, and the underlying whisper repository is installed from GitHub directly rather than from PyPI.

Translation, OCR and speech all share the pipeline

The tool is broader than transcription, and the other three stages are the interesting ones for an overlay use case.

Text translation offers language identification supporting 200 languages, NLLB-200 as a single model with 200 languages and described as high accuracy, M2M-100 as a single model with 100 languages, Meta's multimodal model at about 101 languages, and the Phi-4 multimodal LLM at about 23.

OCR captures game images and translates the in-game text, which is a different problem from speech. EasyOCR supports 80 or more languages, the Phi-4 multimodal model supports about 23 and handles handwriting, and GOT-OCR 2.0 also supports handwriting. The window capture file in the repository, windowcapture.py, is how the image reaches those models.

Text to speech reads the transcriptions or translations aloud. Silero, F5 and E2-TTS with voice cloning and streamed playback, Kokoro TTS with streamed playback, and Zonos TTS with voice cloning and streamed playback are the options.

On top of that sits voice activity detection with Silero-VAD, which decides when someone is actually talking, and retrieval based voice conversion through the RVC plugin, which can convert your voice, the voice in audio files or the voice from text to speech. There is also a large language model stage for continuation of text and automatic answer generation, which the documentation labels a proof of concept, with the Phi-4 multimodal model supporting question answering and extendable function calling, and plugins offering FLAN-T5, GPT-J and Bloomz.

The plugin system is where the extra models live

The plugin architecture is a separate project, with a repository of its own at Sharrnah/whispering-plugins, and the documentation links both a list of available plugins and a plugin creation guide.

That separation is the design decision that makes the core tool manageable. The base application covers the audio capture, the output channels and the core models, and anything beyond that is a plugin. The RVC voice conversion and the LLM options are both reached through plugins rather than being built into the application.

The repository layout backs this up. There is a Plugins directory, a plugin_host.py, a Profiles directory, a Models directory, and a Utilities directory, alongside the core Python modules: audio_processing_recording.py, audio_routes.py, audio_tools.py, audioprocessor.py, remote_audio.py, remote_audio_output.py, transcription_queue.py, settings.py and processmanager.py.

There are also platform-specific requirement files, requirements.amd.txt, requirements.linux.txt, requirements.nvidia.txt, requirements.linux.prereq.txt, and separate start batch files for different tasks, start-transcribe-mic.bat and start-translate-pcsound.bat. The presence of a start-translate-pcsound script is a hint about the use case, capturing the whole computer's sound and translating it.

Running it, and the two gigabyte release problem

The recommended path is a separate native UI application called Whispering Tiger UI, hosted in its own repository, which keeps the Whispering Tiger version up to date and manages the settings more easily.

The standalone releases come with all dependencies included, and the instructions are to go to the GitHub releases page and download from the link in the description, or to find the latest release directly. The parenthetical note explains why the download is not a direct release file: because of the 2 GB limit, there are no direct release files on GitHub.

The rest of the instructions are the practical part. Install CUDA for GPU acceleration, which is recommended, and extract the files to a drive with enough free space, since after downloading a medium Whisper model and a medium NLLB-200 translation model it can take up to 20 GB.

Then you run it using the .bat files, and you edit or copy an existing start file and change the parameters in a text editor for your own command line flags. start-transcribe-mic.bat tries to use your default microphone and is described as a good starting point.

The working-with-the-code and configurations documentation covers the command line flags, the settings file and the source layout, and there is a FAQ and a websocket clients page for people writing their own client.

A local tool with a heavy dependency tree

The dependency list shows what this kind of project costs, and it is worth reading before you install.

The audio stack is there in the form you would expect: sounddevice and soundfile for capture and file handling, PyAudio and PyAudioWPatch for input, pydub and ffmpeg-python for format conversion, resampy for resampling, and librosa. SpeechRecognition 3.10.4 is a dependency too, and python-osc for the OSC output.

The model runtime is heavier. torch is not pinned to a plain version in the visible part of the file, but there is a note that torch-directml is currently not supported with pytorch 2.6.0, bitsandbytes is pinned at 0.49.2, and triton-windows is pinned at 3.4.0.post20 with a set of commented-out direct wheel URLs for other versions, which tells you this is a project that has had to work around Windows packaging problems in the ML stack.

The C library underneath is ctranslate2, which is what faster-whisper uses for fast inference, with sentencepiece and protobuf alongside it. There is also a custom transformers fork referenced in a comment, pinned to a branch, and a custom build of a flash attention runtime shim in flash_attn_runtime_shim.py.

The licence is MIT, the project is not archived, and the release history is steady: v1.3.20.1 on 2026-09-20, v1.3.19.2 on 2026-08-26 and v1.3.19.1 on 2026-08-22, with the last push on 2026-09-20.

Editorial conclusion

Adopt Whispering Tiger if you want live captions for a stream, a VRChat world or a game window without sending audio to a service, because the whole pipeline runs locally once the models are downloaded. Do not adopt it if you want a small install, since the standalone release needs a CUDA setup for GPU acceleration and up to 20 GB of space after the medium Whisper and NLLB-200 models, and the dependency list includes compiled wheels for triton and bitsandbytes. Before you download, check three things: that you have the disk and the GPU for the model sizes you intend to use, that your display application can accept the WebSocket or OSC output, and that the standalone build is what you want rather than the source tree, since the release files are not on GitHub because of the two gigabyte limit.

Frequently asked questions

Official sources

1. Issues
2. License: MIT
3. README
4. Releases
5. Sharrnah/whispering on GitHub