ai-data-extractor: pulling your own AI chat history out of ten tools into one JSONL

The problem: the conversations live in an app's own database

The premise is stated in the first line of the README: extract your own local chat history from AI coding assistants into a single, normalized JSONL format, for fine-tuning, personal analytics, or just backing up years of conversations before an app's local database gets cleared.

That last clause is the reason the tool exists. The conversations a person accumulates with a coding assistant are a genuinely valuable dataset, and they are stored in a place the user does not control: an application data folder, a SQLite database, a set of JSONL files whose naming scheme only the vendor knows. When an assistant is reinstalled, or a workspace is cleared, or a tool is replaced, that history goes. There is no export button, because these are applications rather than services with an account.

So the approach is to go and read the local storage directly, and to make the result uniform so the data is usable rather than merely rescued. The README is explicit about the scope: it is your own local chat history, extracted from your own machine.

The design goal that follows is normalisation. Each tool stores something different, and the extractor converts whatever it finds into one schema, so that a conversation with Cursor and a conversation with Aider end up as the same shape. The README says fields vary a bit by source, since not every tool records code context, token usage or a project path, and that messages, source and session_id are the only ones you can always rely on. That is a useful contract to state, because it tells you what your downstream analysis can assume.

What is captured is more than the text. The feature list covers user messages and assistant responses, code context with file paths, selections and snippets, code diffs and suggested edits where the tool records them, tool calls and their results, and timestamps, session IDs, project paths and model names, whatever each tool's storage actually contains.

Ten tools, and the storage shape behind each one

The supported sources table is the centre of the project, and the third column is the interesting one, because it is where the real work is.

Claude Code stores JSONL, one file per session, under `~/.claude/projects/**/*.jsonl`. Codex CLI stores JSONL rollout files under `~/.codex/sessions/**/rollout-*.jsonl`. Cursor stores SQLite in a file called state.vscdb, under the Cursor user storage location. Windsurf is also SQLite, and the table marks its schema as undocumented with a heuristic. Trae is SQLite plus JSONL, also undocumented and heuristic. Continue uses JSON, one file per session, in ~/.continue/sessions/*.json. Gemini CLI uses JSON, one file per chat, under ~/.gemini/tmp/<hash>/chats/*.json. OpenCode uses JSON with session, message and part trees, under ~/.local/share/opencode/storage/. Cline and Roo Code use JSON with one folder per task, and are marked as new. Aider uses a markdown transcript per project, in <project>/.aider.chat.history.md.

Three formats, ten tools, and the tools differ in how they are identified. Some have a well known folder, some have a file name to look for, and Aider has neither, which is why it needs its own treatment.

The two newest additions come with an explanation of why they were chosen, and the reasoning is worth reading because it is about testing the toolkit rather than about the tools. Cline and its fork Roo Code are described as the most popular open source autonomous coding agent extension, and it stores raw Anthropic-format message arrays per task, so its extractor doubles as the simplest example of parsing that format if you want to add your own tool later. Aider is described as the most popular terminal-only pair programming tool, and it is structured completely differently from everything else here, with no central database, just a markdown transcript sitting in every project directory. It is included specifically to prove the toolkit generalizes beyond the SQLite or JSONL in one app-data folder case.

That is a good reason to include an awkward case deliberately. Aider is the outlier that would otherwise not be supported at all, and adding it proves the search and normalisation layer is not hard-coded to the common shape.

The platform handling is automatic as well. Every script searches macOS, Linux and Windows conventions, covering ~/Library/Application Support, ~/.config, ~/.local/share, %APPDATA% and %LOCALAPPDATA%, and you do not need to tell it which operating system you are on.

Running it, and the flag to use first

The quick start section is short because the tool has no dependencies at all, only the standard library. Python 3.9 or later is required and 3.10 or later is recommended.

The interactive path runs the script and gives you a numbered menu of sources to pick from:

# Interactive: pick which sources to extract from a numbered menu
python extract.py

Driving it directly skips the menu, and there are several useful combinations:

# Or drive it directly
python extract.py --all
python extract.py --sources cursor,claude_code,aider
python extract.py --list                    # just show what's installed, don't extract
python extract.py --all --merge             # also write all_conversations.jsonl

# Shorthand for "extract everything"
./extract_all.sh

The --list option is the one to reach for first. It reports what was found for each source without extracting anything, which the CLI reference calls a fast, safe preview. Given that the tool is designed to sweep your home directory for application data folders, running a preview before it writes anything is the sensible order of operations.

The full set of options is in the reference block:

python extract.py [--all] [--sources ids] [--list] [--output-dir DIR]
                    [--search-path PATH ...] [--merge]

--all                 Extract every supported source, no prompts.
--sources ids         Comma-separated source ids (see table above for names,
                       or run with --list to see them). Skips the menu.
--list                Report what was found for each source without
                       extracting anything -- a fast, safe preview.
--output-dir DIR      Where to write JSONL (default: ./extracted_data)
--search-path PATH    Extra directory to search, on top of the normal OS
                       locations. Repeatable. Mainly useful for:
                         - Aider, which has no fixed app-data folder and
                           needs to know where your projects live
                         - nonstandard install locations for anything else
--merge               After extracting, also concatenate everything into
                       all_conversations.jsonl

The --search-path option exists mostly for Aider, because the markdown transcript lives in the project directory rather than in an application data folder, and the tool has no way to guess where your projects are. It is also the escape hatch for a nonstandard install of anything else.

Each extractor can also be run standalone, the same as the original toolkit, with python -m extractors.cursor from the project root or python extractors/cursor.py, which the README says is handy while debugging one source. The repository layout matches that: a single extract.py, an extractors/ directory, an extract_all.sh script, a README and a LICENSE, and nothing else.

The output format, and the fields you can rely on

Every run creates timestamped files under extracted_data/, one per source, and the merge option adds a combined file:

extracted_data/
├── claude_code_conversations_20260816_143022.jsonl
├── cursor_conversations_20260816_143022.jsonl
├── aider_conversations_20260816_143022.jsonl
├── cline_conversations_20260816_143022.jsonl
└── ... one file per source you extracted, plus all_conversations.jsonl if --merge

The timestamp in the filename is the moment of the run, which means repeated runs accumulate rather than overwrite, and the merged file gives you one stream to work with.

Each line in a JSONL file is one conversation, and the schema is the normalised one:

{
  "messages": [
    {
      "role": "user",
      "content": "How do I fix this TypeScript error?",
      "code_context": [
        {"file": "/Users/you/project/src/index.ts", "code": "const x: string = 123;"}
      ],
      "timestamp": "2026-01-16T14:30:22Z"
    },
    {
      "role": "assistant",
      "content": "The error occurs because you're assigning a number to a string type...",
      "tool_use": [{"name": "edit_file", "input": {"path": "src/index.ts"}}],
      "timestamp": "2026-01-16T14:30:25Z"
    }
  ],
  "source": "cursor-composer",
  "session_id": "c1a2b3...",
  "project_path": "/Users/you/project",
  "name": "TypeScript Type Error Fix",
  "created_at": 1705414222000
}

Two observations about that schema. The source value is not the product name but the shape of the storage, with cursor-composer pointing at a specific format rather than at Cursor in general, which matters because the same product can store conversations in more than one way. And the timestamps inside the messages are ISO 8601 strings while created_at is a numeric epoch value, so a consumer of this data has to handle both.

The message objects carry a role, the content, and optionally code_context, tool_use and a timestamp. The code_context entry pairs a file path with the code that was in context, which is the part that makes this dataset interesting for fine-tuning: it records not just what was asked but what was on screen at the time.

The README is straightforward about the limits of the contract. Fields vary a bit by source, since not every tool records code_context, token usage or project_path, and messages, source and session_id are the only ones you can always rely on.

How it works, and why it never writes to a database

The how it works section is five numbered steps, and they describe a deliberately cautious design.

First, detect the operating system and build a list of plausible data roots, covering Application Support, .config, .local/share, %APPDATA% and the rest. Second, search each root for the tool's known folder names. Third, read the storage, which is JSONL line by line, SQLite via a read-only connection so a running app never blocks us, or JSON trees, depending on the tool. Fourth, normalise whatever is found into the messages schema. Fifth, write JSONL, one conversation per line, into extracted_data/.

The read-only point is the one that matters most in practice. These databases are the live storage of an application that is probably running while you run the extractor, and an application that is running will have the database locked, and a tool that opens it for writing will either fail or, worse, cause the application to fail. Reading only means the extractor works alongside a running Cursor, which is the normal state of a working machine.

The README goes further on the error handling. Nothing here ever opens a database for writing, and every reader is wrapped so that one corrupt or locked file cannot take down the whole run, so you get a partial result and move on rather than a stack trace. For a tool whose job is to sweep a dozen unrelated applications, that is the difference between a useful run and a failed one, since one locked file among ten sources should not cost you the other nine.

The project description calls it a free open-source extractor for AI coding assistant chat histories, and the topics on the repository name the tools it targets along with the general themes of data extraction and data analysis. The licence is MIT, the primary language is Python, the default branch is main, and the last push was on 2026-10-20. There are no GitHub releases, so the tool is distributed by cloning the repository rather than from a package index, which is consistent with its zero-dependency, single-script shape.

The honest limitation: three tools with undocumented schemas

The README has a section at the end devoted to the tools it cannot fully rely on, and it is the most useful thing in the document for anyone deciding how much to trust the output.

Cursor, Windsurf and Trae do not publish their storage schema, and it has changed multiple times. The README says Cursor alone has gone through at least three shapes: a workspace ItemTable chat, an inline composer, and a split bubbleId composer. The cursor.py extractor implements all three known shapes explicitly.

That sentence contains both the strength and the weakness. Implementing three known shapes explicitly means the extractor is not guessing, it is handling documented-by-observation formats, which is much better than a single heuristic. But the formats are not a contract, and a fourth shape will arrive eventually and the extractor will silently find nothing, or find something and misinterpret it.

The Windsurf and Trae entries are weaker still, since the table marks both as undocumented and heuristic. The table entry for Trae describes it as SQLite plus JSONL with an undocumented schema, and the Windsurf entry as SQLite with an undocumented schema, heuristic. A heuristic extractor is one that looks for something that resembles a conversation rather than one that knows where conversations are, so its recall and precision are both lower than for the tools with a known layout.

The same caveat applies to every extractor in a subtler way, because the tools are moving. The tool is a snapshot of storage formats at a point in time, and formats move faster than the tool does, so the practical advice is to treat a recent run as more trustworthy than an old one, and to check the output rather than assume it is complete.

The reason the project includes a section like this at all is that it could have hidden the limitation. A tool that claims to support ten tools and quietly returns half the data for three of them is worse than one that says which three are uncertain. The fact that the README names the specific Cursor formats, so a user can check their own storage and see which shape is in use, is a real diagnostic aid rather than a disclaimer.

Editorial conclusion

Adopt ai-data-extractor if you want your own coding assistant conversations in one place for fine-tuning, personal analytics or backup, and you are comfortable with per-tool storage formats that the vendor does not document. Do not adopt it expecting a general purpose conversation import tool, since each extractor targets a specific local storage layout that can change without notice. Before you run it, use --list to see what is found without writing anything, and read the note about Cursor, Windsurf and Trae, because their schemas are undocumented and the heuristics may need adjusting.

Frequently asked questions

Official sources

1. Issues
2. jatechan/ai-data-extractor on GitHub
3. License: MIT
4. README