SUSI Translator: a streaming speech-to-text API that refuses anonymous access

The client pushes chunks, the server pulls them onto a provider

SUSI Translator is described as a production-ready, real-time speech-to-text and translation HTTP API built with Flask. It accepts streamed audio chunks, processes them asynchronously using a pluggable provider architecture with examples such as OpenAI, Deepgram and Whisper.cpp, and exposes REST endpoints plus a web dashboard for clients to consume the resulting text.

The high-level architecture in the documentation is four steps. Audio sources come first: clients capture audio from a microphone, a local file, a live URL, `stdin`, or YouTube, and continuously POST base64 encoded chunks to the API. Second, a Flask application queues the incoming chunks safely. Third, a background worker pulls chunks from the queue and routes them to configurable external or internal AI providers for transcription and optional translation. Fourth, clients or web dashboards poll the REST endpoints to receive live, sentence-reflowed text.

Two design decisions in that list are worth pulling out. The queue between the API and the provider is what makes the API non-blocking, since a slow provider call does not hold an HTTP request open. And the last step says clients poll, which means the delivery of results is pull based rather than pushed.

The repository layout shows a codebase in transition, with a `flask/` directory holding the backend, a `django/` directory alongside it, a `models/` directory, a `website/` directory, a `benchmarks/` directory, a `gunicorn.conf.py`, and both `pyproject.toml` and a `requirements.txt`. The Python package is named `susi-translator` and the version in the manifest is 0.1.0.

Providers are chosen per session, and overlapping chunks are dropped

The provider registry is the feature that distinguishes this from a wrapper script around one API. The documentation describes it as the ability to dynamically configure the transcription and translation AI models per session, with support for OpenAI, Deepgram and local Whisper models out of the box.

Per session matters in a multi tenant deployment. A classroom using a local Whisper model, a support team using a hosted API, and a broadcast pipeline using a different one can share one server without a global configuration switch, which is what makes the tenant isolation claim credible rather than cosmetic.

The second feature is queue deduplication, described as intelligently deduplicating overlapping audio chunks to optimize API utilization and lower provider costs. This is the part that shows the design is aimed at real audio rather than test signals. A client that pushes 16 kHz mono chunks from a microphone will naturally produce windows that overlap, and a naive implementation pays the provider twice for the same audio. Dropping the overlap before the request leaves is a direct reduction in cost, and it is a cost that scales with every session you add.

The consumption side is where the user experience is decided. The documentation says clients receive live, sentence-reflowed text, and the reflow is what makes the output readable rather than a stream of partial fragments. The dashboard is described as fully featured, for configuring providers, viewing active streams and managing credentials, so the operator side and the client side are both served by the same application.

Anonymous access was removed, and the defaults are safe

The security posture is stated as secure by default, with strict JWT-based authentication, IDOR, or Insecure Direct Object Reference, prevention, and tenant isolation, and no anonymous access permitted. The documentation marks the change as a breaking one: all API clients, including transcription viewing and audio grabbing, now strictly require authentication.

That is a meaningful design decision for a project that started as a prototype, because the easy version of this service is an open endpoint that anyone can post audio to and read transcripts from. The cost is that a client needs a token, which is what the grabber script's `GRABBER_AUTH_TOKEN` environment variable and its `--auth-token` flag are for.

The environment documentation is unusually specific about the production settings, and the comments in `.env.example` state the defaults are safe: loopback only, debug off, CORS limited to localhost. `JWT_SECRET_KEY` is required, must be a strong unpredictable string of at least 32 characters, and the server will refuse to start without it. For production the documentation lists `JWT_COOKIE_SECURE=true`, which requires HTTPS, `JWT_COOKIE_CSRF_PROTECT=true`, `DATABASE_URL` pointing at PostgreSQL rather than the default SQLite, `RATELIMIT_STORAGE_URI` pointing at Redis rather than `memory://`, explicit `CORS_ALLOWED_ORIGINS`, and `FLASK_DEBUG=false`.

The debug line deserves a second look because the comment says it plainly: never combine `true` with a non-loopback host, since the Werkzeug debugger is remote code execution. That is a specific and correct warning, and it appears in both the readme and the example environment file. Rate limiting is described as built in and Redis backed in production, which is the right default for a service where each request can cost money at the provider.

Three commands to get a server and an admin account

The setup path is short. The prerequisites are Python 3.10 or later, `uv` for fast dependency management, and ffmpeg on the `PATH` if you use the file, url or youtube audio sources.

Dependencies are installed with uv, which creates the virtual environment and resolves everything:

uv sync

There is one automatic download to be aware of. The server uses Supertonic 3 for text-to-speech audio generation, and on the first run it downloads around 250MB of ONNX models to `~/.cache/supertonic3`. If you hit rate limits or slow downloads, the documentation says to export a free Hugging Face token as `HF_TOKEN`.

Then the environment is configured by copying `.env.example` to `.env` in the `flask/` directory, and a superuser is created, which the documentation says is required before you can log into the dashboard or configure providers:

uv run python flask/auth/create_superuser.py

The server starts with:

uv run python flask/transcribe_server.py

After that the web dashboard and the Swagger API documentation are available at `http://127.0.0.1:5040`. A separate production consideration is the WebSocket deployment, because when deploying with gunicorn, `flask-sock` requires a WebSocket-capable worker class such as gevent or eventlet instead of a synchronous worker, and the documentation gives the example `gunicorn -k gevent -w 1 transcribe_server:app`.

Migrations must run before the first start

The database handling is one of the more consequential details in the documentation, because getting it wrong produces a server that starts and then fails.

The project uses Flask-Migrate and Alembic for schema upgrades, and the commands have to be run from the `flask/` directory:

cd flask
uv run python -m flask --app transcribe_server.py db upgrade

The documentation spells out two cases. For an existing deployment with a `susi.db` created before migrations were added, the `rooms` table may be missing, and running `db upgrade` adds it without affecting the existing `organizers` or `token_blocklist` data. For a fresh install, `db upgrade` must be run before starting the server for the first time, or the server will fail when it attempts to write room records.

That is the kind of operational detail that separates a prototype from something a team can deploy, and it is documented in a way that does not assume you know the failure mode in advance.

The grabber script lives alongside the server at `flask/audio_grabber.py` and captures audio from one of five sources. The `mic` source uses PyAudio and needs a working input device, `file` uses pydub with ffmpeg, `url` uses ffmpeg, `stdin` is raw PCM passthrough with no extra requirements, and `youtube` uses yt-dlp with ffmpeg.

The usage examples show the shapes:

export GRABBER_AUTH_TOKEN="your_internal_or_session_token"

uv run python flask/audio_grabber.py mic
uv run python flask/audio_grabber.py file --path talk.mp3 --realtime
uv run python flask/audio_grabber.py url --url https://example.com/live.m3u8
uv run python flask/audio_grabber.py youtube --url https://www.youtube.com/live/EXAMPLE_ID

There is also a raw pipe form for anything ffmpeg can read:

ffmpeg -i input.wav -f s16le -ac 1 -ar 16000 - | \
    uv run python flask/audio_grabber.py stdin

The YouTube source has its own caveat. YouTube increasingly returns a sign-in to confirm you're not a bot error, particularly for requests from data-centre IPs, VPNs or WSL, and the documented workaround is to pass cookies from a logged-in session with either `--cookies-from-browser chrome` or an exported cookies file.

Where SUSI Translator is the wrong tool

The limits here are honest to state, because most of them come from the project describing itself accurately.

The first is maturity. The manifest version is 0.1.0, the description calls it a prototype, the repository has no GitHub releases, and the last push to the repository was on 2026-07-31. The word production-ready appears in the readme's own description of the API, and the security work behind that claim is real, but a 0.1.0 package with no release tags is what it is. Pin a commit rather than a version, and expect the configuration surface to move.

The second is the dependency weight. The dependency list includes torch, transformers, faster-whisper, ctranslate2, numpy, scipy, numba, supertonic, soundfile, pyaudio and yt-dlp, alongside the Flask stack, and the Linux x86_64 CUDA packages are pulled in conditionally. A machine that only needs to relay audio to a hosted provider is installing a deep learning stack to do it, and the first run downloads around 250MB of ONNX models.

The third is the delivery model. Clients poll the REST endpoints for live text. For a captioning or transcription client that wants to receive text as it is produced, polling adds latency and needs a poll interval, and the documentation does not describe a server-sent events or WebSocket path for transcript delivery.

The fourth is the YouTube source, which depends on browser cookies and on YouTube not blocking the request. That is a fragile dependency by construction, and the documentation is upfront about it.

The fifth is structural, in the repository layout itself. There are both a `flask/` and a `django/` directory, and the `.env.example` describes its own section as the Flask backend, legacy, which suggests the Django tree is either an earlier iteration or a parallel surface. The documentation does not explain how the two fit together, and a new user has to work that out from the code.

Apache-2.0 terms, a wide dependency surface, and where to look for help

The licence is Apache-2.0, with the `LICENSE` file at the repository root, which is a permissive licence that still requires keeping the copyright and notice information.

The dependency surface is where the operational work sits, and the manifest is honest about it. Alongside the Flask pieces there are the AI stack packages, the WebSocket and admin extensions, the Redis and PostgreSQL clients, `flask-talisman` for security headers, `gunicorn` and `gevent` for serving, `cryptography` and `pyopenssl`, and the `huggingface-hub` client used for the model download. The dev group adds pytest with a Flask plugin and locust for load testing, and the pytest configuration points at `flask/tests`. There is also a `benchmarks/` directory in the repository, and a `gunicorn.conf.py` at the root for the production server.

The `requirements.txt` file is explicitly labelled a legacy fallback only, with the primary setup now being `uv sync` and dependencies managed in `pyproject.toml`, and the file kept for environments that still need pip. The `uv` section of the manifest sets `package = false`, which tells uv this is an application rather than a library to be installed, and that is the right setting for a service that is run rather than imported.

For help, the documentation points to the GitHub issue tracker and to the project community, and the deployment story is spelled out in the readme rather than left to a wiki: environment variables, migration order, the WebSocket worker requirement, and the security defaults are all in the one file. For a project at version 0.1.0, that is a good sign, and it is the reason this service is worth a trial even before the version number moves.

Editorial conclusion

SUSI Translator fits a team that needs live transcription of an audio source into text through an HTTP API, that wants to choose the transcription provider per session rather than hard coding one, and that needs multi tenant isolation with credentials from the start. It does not fit a team that needs a production hardened service today, because the package version is 0.1.0 and the description calls it a prototype, and it is the wrong choice if you cannot accept a dependency tree that includes torch, transformers and faster-whisper plus a 250MB model download on first run. Before you deploy it, set `JWT_SECRET_KEY` to a strong value of at least 32 characters, run the `db upgrade` migration before the first start, and decide whether the polling model fits your client.

Frequently asked questions

Official sources

1. fossasia/voxbento_translator on GitHub
2. Issues
3. License: Apache-2.0
4. README