Heretic: Automated Abliteration That Finds Its Own Parameters
Fully automatic censorship removal for language models
At a glance
- What is it?
- Heretic automates the removal of safety alignment from transformer language models by combining directional ablation with an Optuna-based optimizer. It targets users who want a decensored model without manual tuning or deep knowledge of transformer internals.
- Who is it for?
- Adopt Heretic if you run dense, MoE, or some hybrid transformer models and want to produce a decensored variant without manual parameter tuning. Do not use it if you rely on state-space architectures or if you need a tool that guarantees identical output across hardware.
- Can I use it commercially?
- Yes, with strict conditions. AGPL-3.0 is a network copyleft licence: if people use a modified version over a network, for example as a hosted service, you must offer them its source code under the same licence.
- Is it still maintained?
- Yes. The repository last received commits 10 days ago.
- What is it written in?
- Mainly Python, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
What Heretic Automates and Who It Serves
Heretic solves a specific problem: removing the refusal behavior that safety alignment builds into instruction-tuned language models. This process, called abliteration, normally requires a human to inspect the model's internal activations, pick a direction that encodes refusal, and then surgically ablate that direction. Heretic removes the human from that loop. It finds the ablation parameters itself, using a Tree-structured Parzen Estimator optimizer from Optuna. The tool targets anyone who can run a command-line program but does not understand transformer internals. That includes hobbyists running local models, researchers who need a baseline decensored model, and developers who want to offer an uncensored variant without paying for fine-tuning. The documentation claims it works on most dense models, several MoE architectures, and some hybrid models like Qwen3.5, but pure state-space models are not supported. That scope is broad but not universal, so the first check is whether your model falls inside it.
The Mechanism: Directional Ablation Plus a TPE Optimizer
Heretic builds on directional ablation, the technique described in the Arditi et al. 2024 paper and refined in later blog posts by Lai. The core idea is to find a direction in the model's activation space that correlates with refusal, then suppress that direction during generation. Manual abliteration requires you to choose which layers to modify and how strongly. Heretic instead treats those choices as parameters to optimize. It co-minimizes two objectives: the number of refusals on harmful prompts and the KL divergence from the original model on harmless prompts. The second objective is what preserves the model's general intelligence. If you only suppress refusals, you can damage the model's ability to answer harmless questions. By minimizing KL divergence, Heretic tries to keep the decensored model close to the original except for the refusal behavior. The optimizer uses TPE, a Bayesian method that samples candidate parameter sets and evaluates them against those two metrics. This is a search over a space of possible ablation configurations, not a single fixed recipe.
Getting It Running: Commands and Configuration
The README gives a minimal path to a decensored model. You need Python 3.10 or later and PyTorch 2.2 or later installed for your hardware. Then you run two commands: pip install -U heretic-llm and heretic Qwen/Qwen3-4B-Instruct-2507, replacing that model identifier with your target. The process is fully automatic and requires no configuration. That is the default. If you want control, you can pass command-line options or edit a TOML file named config.default.toml. The repository also uses uv for dependency management and includes a uv.lock file that pins every package version. If you clone the repository, you can run heretic via uv run heretic, which ensures your dependencies exactly match what the developers used. The README recommends this for reliability and security. One specific constraint appears in the PyTorch version note: some models, like gpt-oss with MXFP4 quantization, require torch.accelerator, which was added in PyTorch 2.6. So the minimum version is not always sufficient.
What the Quality Metrics Actually Show
The README includes a comparison table for gemma-3-12b-it. The original model refuses 97 out of 100 harmful prompts. Two human-made abliterations, from mlabonne and huihui-ai, each refuse 3 out of 100. The Heretic-produced model also refuses 3 out of 100. The difference is in KL divergence on harmless prompts: 1.04 for mlabonne, 0.45 for huihui-ai, and 0.16 for the Heretic model. The README interprets this as less damage to the original model's capabilities. That is a plausible reading, but it is not proof of better real-world performance. The README itself says that mathematical metrics are no substitute for human evaluation. The table was compiled on a specific platform, an RTX 5090 with PyTorch 2.8, and the README warns that exact values may vary by platform and hardware. You can reproduce the numbers using the built-in evaluation command, for example heretic --model google/gemma-3-12b-it --evaluate-model p-e-w/gemma-3-12b-it-heretic. That command is useful for verifying claims on your own setup, but it is not a guarantee that the optimizer will find the same parameters on every run.
Limitations and Wrong-Tool Cases
The most obvious limitation is architectural support. Heretic does not support pure state-space models or certain research architectures. If your model is not a dense or MoE transformer, you are out of luck. Another limitation is the automatic process itself. Because it is fully automatic, you have less control over the exact ablation behavior. The optimizer targets a balance between refusal suppression and KL divergence, but that balance may not match your preferences. For example, if you want to preserve maximum capability even at the cost of more refusals, the default objective may not do that. You can tune parameters, but that requires reading the configuration file and understanding what each knob does. The README does not document those parameters in detail. A third limitation is hardware dependence. The README states that evaluation numbers can vary by platform. That means a model that looks great on one GPU might behave differently on another, and the optimizer's search may also be affected. Finally, the tool is not a magic bullet for all safety issues. It removes refusal behavior, but it does not remove other forms of alignment, such as bias or sycophancy, and it may not fully suppress refusals on all prompts, as the 3/100 numbers show.
Alternatives: Manual Abliteration and Other Tools
The main alternative to Heretic is manual abliteration, where a human inspects the model and selects ablation parameters by hand. That is what the mlabonne and huihui-ai models in the README represent. The difference in approach is fundamental. Manual abliteration relies on expert judgment about which layers and directions matter. It can be more precise because a human can look at the model's behavior and adjust accordingly. But it is slow and requires deep knowledge of transformer internals. Heretic replaces that judgment with an automated search. The trade-off is that the optimizer may not find the same quality as a skilled human, but the README claims it can rival human quality on the gemma-3 example. Another alternative is to use a different abliteration tool, such as the ones that produced the huihui-ai models. Those tools may use simpler heuristics or require manual parameter setting. The README references community benchmarks that compare Heretic models favorably, but it does not name the specific tools. If you want to compare, you would need to run Heretic and another tool on the same model and evaluate the outputs yourself. The key difference is automation: Heretic is designed to be run with a single command, while manual methods require you to understand the ablation process.
Maintenance, Licensing, and Community Context
Heretic is licensed under AGPL-3.0. That is a strong copyleft license. If you modify the tool and distribute it, you must release your modifications under the same license. For personal use or internal research, that is not a burden. For commercial products where you redistribute a modified Heretic, it is a serious consideration. The repository is not archived, and the last push was September 2026, with a release v1.4.0 in June 2026. That suggests active development. The README mentions a uv.lock file that pins every package version, which helps with reproducibility but also means you may need to update dependencies manually. The project has a Discord server, a Matrix channel, and a Codeberg mirror, indicating a community around it. The README claims that the community has published over 5000 models with Heretic. That number is not a quality metric, but it does suggest that the tool works for many users. The maintenance cost for you is primarily keeping up with new releases and ensuring your PyTorch version matches the model requirements. The AGPL license is the biggest long-term consideration if you plan to build on the tool.
Editorial conclusion
Adopt Heretic if you run dense, MoE, or some hybrid transformer models and want to produce a decensored variant without manual parameter tuning. Do not use it if you rely on state-space architectures or if you need a tool that guarantees identical output across hardware. Before adopting, verify that your target model is supported and that your PyTorch version meets the model's requirements, especially for MXFP4-quantized models which need PyTorch 2.6 or later. Also confirm that the AGPL-3.0 license fits your distribution plans, since any derivative work you share must carry the same license. Heretic is a practical choice for those who accept these constraints, but its quality claims rest on metrics like refusal counts and KL divergence, which you should reproduce on your own hardware before trusting them.
Community notes