# NotesGPT: a Convex and Whisper voice note app that stops at first record

> Nutlope's template records a voice note, transcribes it with Whisper, and pulls action items out with Mixtral. The README's own setup steps stop one feature short of working.

**Nutlope/notesGPT** — Record voice notes & transcribe, summarize, and get tasks

- Repository: https://github.com/Nutlope/notesGPT
- Website: https://usenotesgpt.com
- Stars: 2,155 · Forks: 370
- Language: TypeScript
- License: MIT
- Published: 2026-10-08 · Updated: 2026-10-08 · Language: en
- Canonical page: https://hysenlabs.com/projects/nutlope-notesgpt

## Record, transcribe, summarize, then throw most of it away

The repository's description is the whole pitch in a few words: record voice notes and transcribe, summarize, and get tasks. The homepage linked from the README is usenotesgpt.com, and the project's own summary line says it generates action items from your notes in seconds, powered by Convex, Together.ai, and Whisper.

That pipeline is four steps. Audio is captured in the browser, stored somewhere, sent to a speech model for a transcript, and the transcript is handed to a language model that produces a summary and a list of action items. The embeddings side exists for a third step, searching across notes.

What makes this a good template rather than just a demo is that each step uses a real infrastructure feature rather than a mock. Convex File Storage holds the voice notes, Convex Vector search handles search, and Convex cloud functions run the backend. That is a coherent answer to a question template projects usually dodge, which is where the file goes and how the search works, and here both have real services attached.

The Future tasks section then tells you what does not exist yet, and it is worth reading in full before you plan around this. Completed action items are currently deleted rather than stored. Recordings are not kept for future playback. Search does not fire automatically as you type. Editing an action item after the fact and setting a due date is an open checkbox. So the demo loop is: record, read the result, lose it.

## The setup steps stop one step before the app works

The Deploy Your Own section is a numbered list of six steps, and step four is the one that catches people. It says that after the frontend and backend are running you should be able to log in but not record. The reason is in steps five and six: you need a Together account and an API key, saved as `TOGETHER_API_KEY` in your Convex environment variables.

The first two steps are the usual ones:

```bash
npm install
npm run dev
```

And the dev script is not a single process. The manifest shows it running two in parallel with npm-run-all, with a `predev` that runs `convex dev --until-success`, a backend script of `convex dev`, and a frontend script of `next dev`. So `npm run dev` brings up the Convex backend and the Next.js dev server together, and the predev hook keeps retrying until Convex is reachable.

The Convex step also creates your project on first run, which is why it prompts for a login. After that it asks for `CLERK_ISSUER_URL`, and the README explains where to get it: make a Clerk account, copy `CLERK_SECRET_KEY` and `NEXT_PUBLIC_CLERK_PUBLISHABLE_KEY` into `.env.local`, follow the Convex Clerk auth steps, and copy the Issuer URL, which looks like `https://some-animal-123.clerk.accounts.dev`, into the Convex environment variables.

So there are two separate environments to configure: `.env.local` for the Next.js side and the Convex dashboard for the backend, and the Clerk issuer URL is the bridge between them. The repository ships a `.example.env` at the root, which is the file to start from.

## npm in the README, pnpm lockfiles in the repository

Here is a discrepancy you will hit in the first five minutes if you follow the README literally. Steps one and two say to run `npm install` and `npm run dev`. The committed lockfile is a pnpm-lock.yaml, and there is a pnpm-workspace.yaml beside it, with no package-lock.json anywhere in the tree.

There is no package-lock.json here, so the lockfile that exists is pnpm's, and there is a workspace file as well. The devDependencies include npm-run-all and the predev script hardcodes `convex`, so nothing forces one or the other, but the committed lockfile is pnpm's and using npm means resolving the tree yourself with no pinned versions.

The concrete thing to check is which lockfile your CI or your deployment platform reads. If you want to reproduce the exact dependency tree the repository records, use pnpm. If you follow the README with npm, you get a tree that was never recorded anywhere, and with a dependency list this size that is a real difference rather than a cosmetic one.

The other version oddity in the manifest is linting. The framework dependency is next at 16.2.6 and react and react-dom are at 19, while eslint-config-next is pinned at 14.0.1. Those two do not belong to the same generation: the Next config for 14 predates both Next 16 and React 19. If you rely on the lint script, expect it to complain about configuration it does not understand, and treat that as a template artifact to fix rather than a signal about your own setup.

There is also no test script in the manifest, and no test directory in the tree. For a template that is normal, but it means the repository's own claim to correctness rests on it running, not on it being covered.

## Which model does what, and where the migration is heading

The tech stack section names the models explicitly, which is helpful because the README's marketing line does not. Together Inference handles the LLM, listed as Mixtral, and handles Whisper transcriptions. Together Embeddings provides the embeddings used for search. So one provider covers all three model calls.

The manifest backs that with together-ai at 0.40.0, and it also carries openai at 4.20.1 alongside @instructor-ai/instructor, which is the structured output library. Having both an OpenAI client and Together's client in the same manifest suggests the structured-output helper is wired through the OpenAI-compatible interface, which Together provides. That is a reasonable arrangement and also a reason to look before assuming every model call goes through one path.

The last item on the Future tasks list is Migrate to incredibly fast whisper, which tells you the transcription model is the part the author expects to change. If transcription cost or latency is what matters for your use of this, that line is the most informative sentence in the repository, because it says the current Whisper configuration is the slow option by the author's own judgment.

On the framework side the manifest is current: Next 16.2.6 with the App Router, React 19, TypeScript 5, Tailwind 3.3 with tailwindcss-animate and a prettier-plugin-tailwindcss for class sorting, Radix UI primitives for avatar, dropdown menu, icons and slot, lucide-react for icons, react-hot-toast for notifications, zod 4 for validation, and next-plausible for analytics.

That Radix plus lucide plus cva combination is the current shadcn-style stack, so the interface is assembled from composable primitives rather than a component library you cannot restyle.

## Reading the repository as a reference implementation

The tree is organized the way a Next.js App Router project with a Convex backend usually is, and the shape tells you where to look. app/ is the App Router, components/ holds the interface pieces, convex/ is the backend with functions and schema, lib/ is shared code, styles/ and the tailwind.config.ts plus postcss.config.js handle styling, public/ holds assets including the open graph image the README links to, and middleware.ts sits at the root.

The middleware file at the root is where Clerk's route protection normally goes, and it is the piece most worth reading first if you are considering swapping authentication, because it defines which routes are public and which require a session.

This is a template with a real product site behind it, which is the shape Nutlope's other projects take. The README points at usenotesgpt.com as the live instance, and the difference between reading the code and using the site is where you decide whether to adapt or to just link to it.

GitHub reports the MIT license, which matches a LICENSE file at the root, 2,158 stars, 372 forks and 21 open issues. The repository is not archived and the last push was on 2026-07-12. There are no published releases, so the commit history is the version history and there is no changelog to read.

There is also a personal detail in the Future tasks list worth noting rather than glossing over: one item is to apply UI updates based on Youssef's redesign, and another asks for smaller text and images on the landing page for multiple screen sizes. A template that names the designer whose work it has not yet applied is a reminder that the code and the design are at different stages.

## What to lift out of it and what to leave

If your interest is the architecture rather than the app, the reusable pieces are specific. The Convex schema plus its file storage calls show how to attach audio files to records and keep them addressable. The vector search setup shows how to index embeddings alongside the same records so search runs over real data. The cloud functions show the pattern for calling a transcription API and then an LLM from inside a backend function, which is the right place for those calls because they need an API key that must not reach the browser.

The Clerk integration is the piece most likely to be replaced. Issuer URL plumbing through a Convex environment variable is a specific shape, and the README's own steps show that it requires three separate pieces of configuration before authentication works.

The parts not worth copying are the landing page, which the Future tasks list says needs smaller text and a nicer look, and the search UX, which is listed as needing to be smoother and to fire automatically as you type.

One last thing to know about the future list, because it affects whether you should treat this as a starting point or a reference: it includes adding a Notion integration to push transcript, summary and action items across, making the app a PWA if there is an easy step, prompt engineering the summary to be more useful than what currently displays, and fixing layout shift on the dashboard by showing a loading state. Four of those are the ordinary work that separates a template from a product.

For a first contact with the code, run the dev script, get the Together key in place so recording actually works, record one note, and then read the Convex function that handles the upload. That path exercises file storage, transcription, the LLM call and vector indexing in one flow, which is more informative than reading the components.

## Conclusion

NotesGPT is a template to read and adapt rather than an application to install, and the Future tasks list says so more honestly than a finished project would. Recording a note, transcribing it and generating action items is a well-chosen demonstration of Convex file storage, Convex vector search and Convex cloud functions working together with Clerk and Together, and the code is organized well enough to lift pieces out of. What it is not is finished: the README's own numbered setup says you will be able to log in but not record until you add the Together key, the notes are not kept for playback, and the search experience is listed as needing work. The mismatch to check before you build on it is tooling, not architecture, since the README asks for npm while the repository ships pnpm lockfiles and a pnpm workspace file, and the lint stack pairs eslint-config-next 14 with Next 16. Copy the three service integrations and the Convex schema shape, supply your own auth if Clerk is not a fit, and treat the landing page work and PWA idea as unfinished business.

## FAQ

### Why can I log in but not record a note in NotesGPT?

The README's own setup steps say this is expected after the frontend and backend are running. Recording needs the transcription and language model calls, so you have to create a Together account, get an API key, and save it as TOGETHER_API_KEY in your Convex environment variables. Once that key is in place the recording path works.

### Which AI models does NotesGPT use?

The tech stack section names them. Together Inference serves the LLM, listed as Mixtral, and also handles Whisper transcriptions, and Together Embeddings supplies the vectors used for search. The manifest carries together-ai at 0.40.0 along with an OpenAI client and the instructor library for structured output.

### Do I need Clerk to run NotesGPT myself?

Authentication is wired through Clerk, and the README requires a Clerk account, the CLERK_SECRET_KEY and NEXT_PUBLIC_CLERK_PUBLISHABLE_KEY in .env.local, and the Clerk Issuer URL added to your Convex environment variables. The middleware file at the repository root is where route protection is defined, which is the place to change if you want different authentication.

### Are recorded voice notes and completed action items saved?

Not yet, according to the Future tasks list. Keeping recordings for future playback and displaying them on the page is an unchecked item, as is storing completed action items instead of fully deleting them. Editing an action item afterwards and setting a due date is also listed as unfinished work.

## Sources

- [Issues](https://github.com/Nutlope/notesGPT/issues)
- [License: MIT](https://github.com/Nutlope/notesGPT/blob/main/LICENSE)
- [Nutlope/notesGPT on GitHub](https://github.com/Nutlope/notesGPT)
- [Project website](https://usenotesgpt.com)
- [README](https://github.com/Nutlope/notesGPT/blob/main/README.md)

---

Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/nutlope-notesgpt