StoryToolkitAI: AI transcription and footage search for editors who would rather not leave the timeline

What StoryToolkitAI solves for editors with too many hours of footage

The problem is not transcription. It is finding the one sentence in forty hours of interviews where someone says the thing you half remember. StoryToolkitAI transcribes footage with OpenAI Whisper on the local machine, indexes it, and lets you search by meaning rather than exact words. The README describes the tool as one that "tries to understand your footage and helps you edit more efficiently with the assistance of AI", and the feature list backs that up: full video indexing and search, automatic speaker detection, transcript groups, and exports to SRT, TXT, AVID DS and Fusion Text nodes.

The intended user is a film or documentary editor, not a machine learning engineer. The author writes that he is a filmmaker who hates to code and that his team uses the tool daily in their editing room. That origin explains both the feature set and the roughness: features arrive because a working cutting room needed them, and the README warns the tool is "still being actively developed, raw and unpolished".

It runs standalone, independent of any editing software, but it also integrates with DaVinci Resolve Studio 18+, where it can mark and navigate timelines via transcript, search timeline markers with AI, copy markers between the transcript and the timeline, and import subtitles directly into a Resolve bin. If you cut in Resolve Studio, that integration is the reason to look at this over a plain transcription script.

How the transcription, indexing and story pipeline actually fits together

The pipeline is a chain of local models. Whisper handles speech recognition. Sentence Transformers produce embeddings that make semantic search possible, so a query about a topic can match a line that never uses your words. spaCy and spacy-langdetect handle language detection and text processing. Speaker detection uses pyannote.audio and speechbrain, according to requirements.txt and the feature list, which is why the speaker features arrived as a beta in v0.23.x.

On top of that sit the language model features. The tool is compatible with OpenAI, ollama, vLLM and LM Studio, so the Assistant can run against a hosted model or a local one. Stories and Selections can be generated by asking an LLM to pick from your footage, and transcripts can be translated to other languages using OpenAI GPT. The output of a story is not a rendered video: it is an EDL, XML or Fountain file you import into your editor. That is the key architectural decision. StoryToolkitAI does not try to become an NLE; it produces edit decision lists and lets Resolve or another editor do the cutting.

Data flow is deliberately local. The README states that the tool only sends data to the Internet in three cases: the API key check against storytoolkit.ai when you enter it in Settings, the Assistant when you send context to OpenAI or another external provider, and an update check on every start. Transcription and translation run on your machine. If you point the Assistant at a local model through ollama or LM Studio, the external calls shrink to the API key check and the update check.

Installing StoryToolkitAI from source and running a first transcription

The README points to the releases page for the latest standalone build, but adds that standalone releases will most likely always be behind the git version. If you want the newest features, the project recommends installing from source, with detailed steps in INSTALLATION.md. The repository is Python, and requirements.txt is where the real dependency weight lives: torch, torchaudio, torchvision, openai-whisper pinned to a specific git commit, sentence_transformers, pyannote.audio, speechbrain, spacy, opencv-python, moviepy and a fork of CustomTkinter. Expect a large download and a machine with a decent GPU or a lot of patience on CPU.

The requirements file is the only installation artifact the repository gives in full, and it is a list of packages rather than a sequence of commands. Its exact contents are what you install against:

wheel
certifi>=2023.7.22
charset-normalizer==2.1.1
ffmpeg-python==0.2.0
filelock==3.8.0
future==0.18.3
huggingface-hub>=0.10.0
idna>=3.7
scikit-learn
librosa==0.10.1
more-itertools>=8.14.0
numpy
numba>=0.59.1

Several entries are pinned to exact versions, including charset-normalizer==2.1.1, filelock==3.8.0, librosa==0.10.1 and numba>=0.59.1. That mix is a compatibility surface: if you install into an environment that already has conflicting versions of torch or transformers, you will spend your evening resolving it. A clean virtual environment is not optional here.

Further down the same file, three dependencies come from git rather than from a package index:

openai-whisper @ git+https://github.com/openai/whisper.git@517a43ecd132a2089d85f4ebc044728a71d49f6e
clip @ git+https://github.com/openai/CLIP.git@a9b1bf5920416aaeaec965c25dd9e8f98c864f16
customtkinter @ git+https://github.com/octimot/CustomTkinter.git@a2a8c37dd8dac1dee30133476596a5128adb0530

That means installation depends on those repositories remaining reachable and unchanged. Once installed, the workflow described in the README is: transcribe footage, let the tool index it, then search. Transcription and indexing are the free local features. If you want the Assistant, you enter an API key in the Settings Window or configure a local provider such as ollama. From there you can search content without typing exact words, group transcript lines, and export to SRT, TXT, AVID DS or a Fusion Text node. The README also links a how-to video for full video indexing and search, which is the fastest way to see the intended flow.

Where StoryToolkitAI gets in your way

The most honest limitation is stated by the author: the tool is raw and unpolished. That is not false modesty. It is a desktop application built on a forked CustomTkinter, maintained by a working editor, and the README itself notes that some features are only available in the non-standalone version and will reach the standalone version "in the next release". If you depend on the standalone build, you are depending on a build that lags the source.

Dependency fragility is the second constraint. requirements.txt pins openai-whisper to a git commit, CLIP to a git commit, and CustomTkinter to a fork at another commit. Git-sourced dependencies mean installation depends on those repositories remaining reachable and unchanged. On a machine with an existing ML stack, version conflicts are likely rather than possible.

Hardware is the third. Whisper, Sentence Transformers, pyannote.audio and speechbrain all run locally. That is the privacy advantage and the performance cost. On a laptop without a capable GPU, indexing a long documentary will be slow, and the tool gives you no cloud fallback for transcription by design.

Finally, know when it is the wrong tool. If you only need subtitles for a single short clip, a lightweight Whisper wrapper is less setup. If you need frame-accurate multicam conforming, that is your NLE's job, and StoryToolkitAI only hands it an EDL or XML. If you need a tool with a formal support contract, this is a Patreon-funded project, not a vendor.

StoryToolkitAI compared with a plain Whisper transcription script

The obvious alternative is running Whisper directly, either the openai-whisper CLI or a wrapper like whisper.cpp, and searching the resulting text files yourself. The difference in approach is real. A Whisper script gives you a transcript. StoryToolkitAI gives you a transcript plus embeddings, which is what makes semantic search work: Sentence Transformers let you find a passage by topic instead of by the exact phrase someone used. It also layers speaker detection, transcript groups, question detection and story generation on top, and it exports to formats an editor can import.

That extra layer costs you the simplicity of a single command. Whisper alone has almost no dependency surface compared with torch, pyannote.audio, speechbrain, spaCy and CLIP in one requirements file. If your actual need is a subtitle file, the heavier tool is the wrong trade.

The other comparison is with cloud transcription services. They require uploading your footage, which the README explicitly avoids for sensitive content. StoryToolkitAI keeps transcription and translation local, at the cost of your own compute and a more involved install. For documentary work under privacy constraints, that trade is usually the right one.

Maintenance, releases and what GPL-3.0 means for your project

The repository is not archived and the last push was on 2026-07-28, so the source tree is being touched. Release history tells a slower story: v0.25.1 landed on 2025-02-20, v0.23.2 on 2024-01-26, and v0.23.1 two days earlier. Between the last tagged release and the last push there is a long stretch of unreleased work, which matches the README's warning that standalone builds lag the git version. If you install from source you get the newest code; if you install a release you get something older and more settled. Upgrade cost is dominated by dependencies rather than by the application code. Because requirements.txt pins exact versions and pulls three packages from git, an upgrade can mean rebuilding the environment rather than running a package manager update.

The licence is GPL-3.0. For an individual editor or a studio using the tool internally, that is normally unproblematic. The implication to check with your own counsel is distribution: if you ship a product that incorporates GPL-3.0 code, the licence's obligations attach. StoryToolkitAI is an application you run, not a library you link into a closed product, but if you plan to bundle or modify it for redistribution, read the LICENSE file in the repository rather than assuming. Nothing here is legal advice.

Editorial conclusion

StoryToolkitAI fits editors and small post teams who want local transcription and semantic search without uploading rushes, and who are willing to install from source and tolerate a tool its author calls raw. It is a poor fit if you need a stable GUI, guaranteed release cadence, or a permissive licence for closed-source redistribution, since it is GPL-3.0. Before adopting it, verify that your Python and torch install matches requirements.txt, that the standalone release is far enough behind the git version for your needs, and that your machine can run Whisper and the indexing models locally.

Frequently asked questions

Official sources

1. Issues
2. License: GPL-3.0
3. octimot/StoryToolkitAI on GitHub
4. README
5. Releases