# DataClaw: publishing agent chat history with redaction and consent gates

> DataClaw turns Claude Code and Codex session logs into Hugging Face datasets, with six redaction layers and a flow that refuses to publish until you hand over your full name and sign three attestations. The redaction is real, but the last check is left to you.

**peteromallet/dataclaw** — Agent harness to publish your agent chat history as Huggingface datasets. 

- Repository: https://github.com/peteromallet/dataclaw
- Stars: 2,123 · Forks: 228
- Language: Python
- License: MIT
- Published: 2026-10-13 · Updated: 2026-10-13 · Language: en
- Canonical page: https://hysenlabs.com/projects/peteromallet-dataclaw

## Session logs become dataset rows, minus the parts you exclude

DataClaw is a Python package that treats your own coding-agent history as data meant to be shared in the open. The one-line description in pyproject.toml reads "Export your coding agent conversations to Hugging Face as structured data", and the keyword list underneath names claude-code, codex, gemini-cli, opencode and openclaw as the sources it knows about. It is aimed at developers who want a Hugging Face copy of their own sessions, not a transcription of somebody else's work.

The export has six parts. User messages come first and carry voice transcripts and images along with the typed text. Assistant responses follow. Assistant thinking is written out too, and --no-thinking turns that off. Tool calls are recorded with tool name, inputs and outputs. Token usage is split into input and output tokens per session, and metadata records the model name, the git branch and timestamps.

Two of those parts shape everything else. Because tool inputs and tool outputs are exported, anything an agent read out of a file, a shell or an environment variable can end up in the dataset, and because thinking blocks ship by default, internal reasoning goes along unless you opt out. That is the reason the tool asks for a source scope and a project scope before it writes anything, and the reason an export is never a single command. Every dataset goes up carrying the dataclaw tag, which is how the project intends separate uploads to add up to one distributed collection.

## Six redaction layers and the disclaimer printed under them

Redaction runs in six layers before anything leaves your machine. Your OS username and any usernames you configure are replaced with stable hashes. Regex patterns catch JWT tokens, API keys for Anthropic, OpenAI, Hugging Face, GitHub and AWS, plus database passwords, private keys and Discord webhooks. Entropy analysis flags long high-entropy strings sitting in quotes as possible secrets. A separate pattern catches email addresses, you can add your own strings to the list, and tool inputs and tool outputs get the same treatment as ordinary messages.

Straight under that list the README prints "This is NOT foolproof" and names what the layers miss: service-specific identifiers, third-party PII, and secrets in unusual formats. The same paragraph points at a follow-up path instead of asking for trust. Convert the export with dataclaw jsonl-to-yaml, then take the result to trufflehog or gitleaks.

The difference in approach is where each one acts. DataClaw rewrites values while it builds the file, so a username leaves as a hash and a matched secret is replaced in place, and the original string is simply not in the artifact afterwards. trufflehog and gitleaks read a finished file and report what is still inside it. One pass changes the data, the other grades it, which means the scanners add coverage only for what the patterns missed and cannot tell you about a value already swapped out.

## The flow that will not publish until three attestations are signed

The prompt shipped with the project is written for an agent rather than a human, and it is the clearest description of how the tool behaves. The steps are fixed: install, install the skill, prep, choose the source scope, choose the project scope, set redacted strings, export locally, review, publish. Four of them carry an uppercase marker, three saying REQUIRED BEFORE EXPORT or REQUIRED BEFORE PUSH and the publish step saying ONLY AFTER EXPLICIT USER APPROVAL.

Scope gets asked for twice, once per source and once per project. dataclaw config --source all sets the source scope, then dataclaw list --source all prints every project with its name, source, session count, size and excluded flag, and the instruction is to send that whole list to the user before anything is filtered. Exclusions then come from dataclaw config --exclude "project1,project2" or from dataclaw config --confirm-projects, and dataclaw config --redact "string1,string2" plus dataclaw config --redact-usernames "user1,user2" follow the same ask-then-set shape.

The local export writes a file and touches no network:

```bash
dataclaw export --no-push --output dataclaw_export.jsonl
```

Publication is a single flag carrying a sentence in it, and the review step before it wants your legal name next to three attestations:

```bash
dataclaw confirm --full-name "THEIR FULL NAME" --attest-full-name "..." --attest-sensitive "..." --attest-manual-scan "..."
```

```bash
dataclaw export --publish-attestation "User explicitly approved publishing to Hugging Face."
```

There is one rule aimed straight at an automated caller: never run a bare hf auth login, always pass --token. Credentials travel in the command line rather than through an interactive login.

## Terminal install and the prep command that has to come first

The CLI is the supported path for terminal work, for Intel Macs, and for anyone driving DataClaw from a coding agent. It installs the usual way:

```bash
pip install -U dataclaw
```

If that fails, the shipped prompt falls back to a clone and an install from the checkout:

```bash
git clone https://github.com/peteromallet/dataclaw.git /tmp/dataclaw && pip install -U /tmp/dataclaw
```

The package needs Python 3.10 or newer and declares four runtime dependencies with floors at huggingface_hub 0.35.0, orjson 3.0.0, pyahocorasick 2.3.0 and PyYAML 6.0.0. PyYAML is the only one whose use the README names, in the dataclaw jsonl-to-yaml conversion step. The console entry point is dataclaw.cli:main, so the command you type and the package you import share one name.

After install, prep comes before any export:

```bash
dataclaw prep
```

That command also sets the contract for the whole session, because every dataclaw command prints next_steps and next_command in its JSON output and the instruction to the agent is to follow those fields rather than improvise. A caller that reads JSON decides what to do next, which is also why a human running the CLI by hand gets a flow built out of flag combinations instead of a single interactive screen.

## An unsigned Apple Silicon app beside an x86_64 build target

The Mac app is a menu-bar build for Apple Silicon Macs, delivered as a DMG that contains DataClaw.app and needs nothing else on the machine. It ships unsigned and the first launch costs about 20 seconds of Gatekeeper friction: double-click the app, dismiss the block dialog with Done rather than Move to Trash, open System Settings, search for Privacy & Security, scroll to Security, click Open Anyway, then confirm with Touch ID or a password. Apple Developer ID signing is being set up, so every new user walks through that once and later launches are normal.

Underneath, the app is two pieces. The Makefile drives a Tauri build in the app directory through pnpm -C app tauri build, and a Python sidecar built by bash scripts/build-sidecar.sh arm64 with pyinstaller.spec driving the frozen executable. The local bundle target passes APPLE_SIGNING_IDENTITY=- and sets createUpdaterArtifacts to false, which is the direct reason there is no updater and no signature in what you download.

Here is the detail a skim misses: the same Makefile also defines an x86_64 sidecar target that hands x86_64 to the build script, while the installation instructions only ever point at an Apple Silicon DMG and steer Intel Mac users to the CLI. The code path for an Intel build exists in the repository, and the project does not say where such an artifact would be published.

## A redaction tool that asks for your full name

Six automated layers, and then three attestations anyway, because the project does not treat its own redaction as sufficient evidence. The review step asks for a full name so it can run an exact-name privacy scan against the export before publication, and the confirmation command carries that name beside an attestation about the name scan, an attestation about sensitive content, and an attestation about a manual scan you performed yourself.

Declining is possible, but not silently. --skip-full-name-scan exists for people who will not type their legal name into a tool, and the flow then requires the reason to be attested rather than letting the check pass empty. The design is candid about its own limits and strange in the same breath: a privacy tool whose strongest gate needs the exact identifier it exists to hide. The documentation does not say whether that name is written into the dataset, kept only for the duration of the scan, or left in the export file, so this is the one place where the commitment cannot be checked from the outside before you run it.

## Claude Code gets a skill, every other agent gets JSON

Step 2 of the shipped prompt installs a skill and states the limit in one line: skill support is currently only available for Claude Code. The command is dataclaw update-skill claude. For any other agentic tool the instruction is to skip that step entirely, not improvise a custom flow, and instead follow each DataClaw command's own output, especially next_steps and next_command.

That split explains the layout. Claude Code users get a skill file, and the repository carries a .claude directory alongside an AGENTS.md for everyone else, together with docs, scripts and tests directories. Every other client is expected to be a program that reads JSON, decides, and runs the next command, which matches the credential rule written for automation: never a bare hf auth login, always --token on the upload. It also means an agent can be interrupted at any gate and resumed from the blocked step, since every command restates the checklist rather than carrying hidden state forward.

## What the packaging metadata fixes and leaves to an extra

pyproject.toml pins the build to setuptools 77.0.0, marks the package version 0.5.1, and classifies it as Development Status :: 4 - Beta, which fits a tool that parses log formats written by other people. Linting runs through ruff with a 120 character line limit and import sorting switched on, and the dev extra carries pre-commit, pytest and ruff.

The optional extras decide what actually ships inside the Mac app. The build extra adds pyinstaller 6.10, keyring 25 and secretstorage 3.3 on Linux, which together turn the CLI into a frozen sidecar and hold credentials in the system keychain. The pii extra pulls in transformers 4.57, torch 2.3, accelerate and tokenizers for an optional model-based PII filter, and the comment above that list is explicit: it is deliberately not a core dependency and not bundled in the Mac sidecar by default. A pytest marker named pii skips those tests when the stack is missing, so the default test run never pulls a model.

On upkeep, the package carries an MIT licence and the last push on the default branch main was 2026-06-05. Upgrade cost sits in two places: the huggingface_hub floor and the upload API beneath it, and the session log formats of whichever agent you point the tool at, which the project pins to no version at all.

## Conclusion

DataClaw fits a developer who already knows what sits inside their own agent logs and wants a structured copy of that history on Hugging Face. It does not fit a team that wants one button to publish a shared account's sessions, because the flow demands your full name, three attestations and an explicit approval before a single upload. Export once with --no-push first, read the jsonl for voice transcripts, images and thinking blocks, and confirm your agent's log format is one the parser actually handles.

## FAQ

### Does DataClaw upload my conversations to Hugging Face without asking me?

No. The flow requires a local export with --no-push, a review step with three attestations, and a separate publish step that carries an explicit approval attestation. The shipped prompt states that publication happens only after explicit user approval.

### What does DataClaw need before it can run?

The CLI needs Python 3.10 or newer and installs with pip install -U dataclaw. The Mac app bundles its own dependencies and is offered only for Apple Silicon Macs, and its first launch requires the Gatekeeper Open Anyway path because the build is unsigned.

### Can I use DataClaw with Codex or another agent, not just Claude Code?

Yes. The package keywords list codex, gemini-cli, opencode and openclaw alongside claude-code, and the source scope is chosen explicitly with dataclaw config --source all. Skill installation is the Claude Code only part, so other agents are told to skip it and follow the next_steps and next_command fields instead.

### How do I check an export before anything is published?

Export locally with dataclaw export --no-push --output dataclaw_export.jsonl, then review the file. The documented follow-up is dataclaw jsonl-to-yaml followed by a pass through trufflehog or gitleaks, and the project states plainly that its own redaction is not foolproof.

### What happens to my assistant thinking blocks and images in the dataset?

Assistant thinking is exported by default and can be dropped with --no-thinking. User messages keep their voice transcripts and images, and tool calls are exported with their inputs and outputs, so anything an agent read from your machine can travel with them.

## Sources

- [Issues](https://github.com/peteromallet/dataclaw/issues)
- [License: MIT](https://github.com/peteromallet/dataclaw/blob/main/LICENSE)
- [peteromallet/dataclaw on GitHub](https://github.com/peteromallet/dataclaw)
- [README](https://github.com/peteromallet/dataclaw/blob/main/README.md)
- [Releases](https://github.com/peteromallet/dataclaw/releases)

---

Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/peteromallet-dataclaw