Nomily AI: a bring your own key transcription and summary app for voice recorders

This repository is documentation and a reference service, not the app

The first thing to understand about Nomily-Ai/nomily-app is that it is the project entry point, not the application. The runnable native clients live in separate repositories, nomily-ios for iPhone and Apple Watch in SwiftUI and nomily-android for Android in Jetpack Compose.

The readme is explicit about the boundary. This repository is not an installable client or a Docker deployment, and it tells you to read the client specific build instructions before starting a local build. The top level entries match that description: a README in English, Simplified Chinese and Traditional Chinese, LICENSE, SUPPORT.md, CONTRIBUTING.md, CODE_OF_CONDUCT.md, SECURITY.md, an examples/ directory and media/ with the diagrams.

What the examples directory holds is the local ASR reference server, which is described as implementing the exact Local ASR Server contract. It needs Python 3.10 or newer and ffmpeg, and the readme gives the macOS and Debian install hints.

brew install ffmpeg

So the repository is best understood as documentation plus a reference implementation of one contract. If you want to run the app on a phone, you are using the App Store build or the hosted APK. If you want to build it yourself, you go to one of the two client repositories. If you want to supply your own speech recognition, you run the reference service here.

One repository detail is worth flagging for anyone assessing the project. The primary language is listed as unknown, which is unusual, and the topics are split between Kotlin and SwiftUI for the clients and asr, diarization and whisper for the transcription side. This is a multi repository product where the entry point itself is mostly prose and an example.

Recorder to client to your own key, with no shared cloud in the middle

The data flow diagram in the readme is the clearest statement of the design, and it is worth reading as the whole architecture.

V05E recorder / Apple Watch / audio imported on iPhone
                  │
                  ▼
        Nomily AI native client
  local library · transfer · playback
                  │
        ┌─────────┴──────────┐
        ▼                    ▼
 Azure Speech or        Your LLM provider
 local ASR endpoint     summary · title · translation
 diarization when       OpenAI · Anthropic · Gemini
 Azure is selected      OpenRouter · Ollama · compatible endpoint
        │                    │
        └─────────┬──────────┘
                   ▼
         Transcript and summary

The client sits in the middle. It holds a local library of recordings, handles transfer and playback, and is the only component you install and operate. Everything to the left is where audio comes from, everything to the right is where it goes for processing.

The framing in the readme is that Nomily AI is not a hosted transcription account or a shared recording cloud. It is a native companion workflow for recordings and user configured AI services, and it keeps the choice of transcription and summary provider with the user.

The privacy section makes the same point as a table of who sends what. The client layer sends the recording library, transfer state and generated files, and you operate that. Azure Speech receives audio for transcription when Azure is selected, under your Azure account and key. A local ASR endpoint receives audio when you select one, and you operate the endpoint. The LLM provider receives transcript text needed for a summary, title or translation, under your selected provider, model, endpoint and key.

Nothing in that table is Nomily operated, and the readme says so in as many words. It also states that the public repositories do not include a shared Nomily AI cloud, a bundled provider account, or a complete self hosted sync stack.

Two transcription routes, and diarization only on one of them

Recording import covers the V05E BLE recorder workflow, where the app connects to the recorder, downloads recordings and manages a local recordings library. On iPhone it also imports recordings transferred from the Apple Watch companion.

Transcription happens live or on recorded audio, and there are two routes. The first is Azure Speech. The second is a user operated local ASR endpoint that is faster-whisper compatible. The distinction is not just about cost, it changes what you can get back.

Azure transcription enables speaker diarization for up to 10 speakers. The local route does not get that. So if your use case is a meeting or a multi-person conversation where knowing who said which line matters, the local endpoint alone will not give it to you, and you need the Azure path.

That is a genuine constraint, and the readme is upfront about it. The privacy framing also has a trade-off here: diarization requires sending the audio to a provider who operates the service, while the local endpoint keeps the audio on a machine you control but gives up the speaker labels.

Summaries, titles and translations are generated by a configured LLM provider, and the list is long: OpenAI, Anthropic, Gemini, OpenRouter, Ollama, or a custom OpenAI compatible endpoint. Note what the LLM receives in that pipeline. It is transcript text, not audio. So even with a cloud transcription provider, the summary step does not require re-uploading the recording.

The readme also states that local ASR and Ollama are provider integrations, not a claim that every product layer is self hosted. That sentence is doing real work, because it is the difference between a privacy feature and a privacy guarantee.

Running the local ASR service on your own network

The advanced section covers the case where you want both speech recognition and the language model running on hardware you control, and the guidance is specific about the network topology. Run both services on a trusted computer on the same private Wi-Fi as the phone, and do not expose either service to the public internet.

That last instruction is the important one. A local ASR server with no authentication is an open transcription endpoint for anyone who can reach it on your network, and a local LLM endpoint is an open inference endpoint. Putting them on a private network is the mitigation, and it is also why the readme frames this as an optional advanced setup rather than a recommended default.

The contract is fixed and specific. The reference service implements the exact Local ASR Server contract, with a WebSocket port 12300 for live audio and an HTTP port 12301 for recording files.

Two ports for two very different needs. Live streaming audio over WebSocket is the hard case, because the client has to push frames as they arrive and get transcripts back incrementally. Handling whole recording files over HTTP is the easy case, a request and a response.

Because the contract is exact, the reference implementation is not just an example, it is a specification you can implement against. If you run whisper or any other engine that does not match this contract, you have an adapter to write. The readme explicitly says this is an optional advanced setup and not a complete self hosted Nomily stack, so what you get is the two endpoints, not a server that also stores, syncs and serves the library.

The dependency is light: Python 3.10 or newer and ffmpeg. Nothing in the visible documentation suggests a database or a container image is involved, which keeps the local setup small enough to run on a laptop that also has to run the phone's Wi-Fi.

Licensing, distribution and the limits of the promise

The licence field on the repository says Other, a custom licence GitHub cannot classify, and the badge on the project page says License: Source available. The distinction from an open source licence is the point. Source available means you can read the code and build it, and the terms are set by the project rather than by an open source licence body. The readme directs anyone with licensing questions to SUPPORT.md and the [email protected] address, which is the right route rather than guessing.

Distribution is split by platform in a way that creates friction. The iPhone app is on the US App Store, the Android build is distributed as a direct APK download with Google Play review in progress, and the readme tells users to use that link until the store listing is approved. Store availability may vary by storefront and version.

The readme also states that source commits do not promise device or firmware compatibility or a production service release. For a project whose client talks to a Bluetooth LE recorder, that sentence is doing necessary work: the hardware compatibility surface is controlled by firmware on a device you do not own, and a commit in the repository is not a guarantee that a given recorder revision behaves the same way.

The other thing worth saying plainly is the absence of a team workspace. There is no shared recording cloud, no bundled provider account, no complete self hosted sync stack. The privacy story is genuinely good, and it comes at the cost of the collaborative features a hosted product would provide. For one person recording their own meetings or interviews, that is a reasonable trade. For a team sharing a library of recordings across devices, it is not, and no amount of configuration inside this repository will change that.

Editorial conclusion

Nomily AI suits a user who records conversations on a V05E or Apple Watch and wants transcripts and summaries without handing their audio to a hosted transcription account, because the ASR and LLM providers are chosen and paid for by the user and a local ASR endpoint is supported. It does not suit someone who wants a shared team recording library with sync, because the documentation states plainly that the public repositories do not include a shared cloud or a complete self hosted sync stack. Before adopting it, read SUPPORT.md and the client specific build instructions, and note that the licence is source available rather than open source, that the Android APK is hosted outside Google Play while Play review is in progress, and that this repository holds documentation and the local ASR reference service rather than an installable client.

Frequently asked questions

Official sources

1. Issues
2. Nomily-Ai/nomily-app on GitHub
3. Project website
4. README