Heretic: refusal removal by default, scorer plugins and an Optuna search for everything else

Passing only a model name means refusal suppression

Heretic is a command line tool published on PyPI as heretic-llm. It needs a Python 3.10+ environment with PyTorch 2.6+ already installed for your hardware, and then a single command against a Hugging Face model id:

pip install -U heretic-llm
heretic Qwen/Qwen3-4B-Instruct-2507

You replace Qwen/Qwen3-4B-Instruct-2507 with whatever model you want to process. The important detail is what happens when you give it nothing else: the default action, defined in config.default.toml, is censorship removal, meaning refusal suppression. There is no prompt asking which behavior you want to change. If your goal is less slop or less sycophancy rather than fewer refusals, you have to supply a different configuration before the run, otherwise you spend the afternoon producing a model you did not ask for.

One config.toml in the working directory decides the whole run

Heretic reads its configuration from config.toml in the current working directory, or from the path stored in the HERETIC_CONFIG_FILE environment variable. The repository ships ready-made alternatives next to the default: config.noslop.toml to reduce slop, config.nohumor.toml to reduce humor, config.nosyco.toml to reduce sycophancy and config.emotions.toml to alter emotional alignment. Two more files, config.classifier.toml and config.piqa.toml, sit at the top level of the repository as well. Because the lookup is tied to the working directory, launching heretic from a folder that happens to contain an unrelated config.toml changes what the tool does. Setting HERETIC_CONFIG_FILE explicitly is the safer habit when you keep several profiles. The full list of options lives in config.default.toml itself, which doubles as the reference documentation for the settings.

Heretic 2.x will not read a 1.x configuration file

Version 2.0.0 was tagged on 2026-11-01, after v1.4.0 in June 2026 and v1.3.0 in May 2026. The project states plainly that configuration files written for Heretic 1.x are not compatible with Heretic 2.x. The consequence is concrete: running pip install -U heretic-llm on a machine where you kept a tuned 1.x config upgrades you across that boundary, and the old file stops working. You have to rewrite it following docs/migration_v1_to_v2.md before the next run. Anyone who shares configs with a team, or publishes them next to models on Hugging Face, should note which major version each file targets. Pinning the package to a 1.x release is the other option if you cannot migrate yet. Development continues past the release: pyproject.toml on master already reports version 2.1.0.dev0, and the last push landed on 2026-11-01.

Scorers define what the optimizer is chasing

Heretic steers a model by optimizing one or more scores, and each score comes from a scorer plugin. Out of the box it measures refusal rate, implemented in src/heretic/scorers/keyword_rate.py, together with KL divergence from src/heretic/scorers/kl_divergence.py. The pairing matters: one score pushes refusals down while the other tracks how far the modified model drifts from the original. The scorers setting in the configuration file chooses which plugins are active and whether each score is minimized or maximized. Any number of scorers can run together and are co-optimized, and the same scorer can appear several times with different settings. To write your own, inherit from the Scorer abstract base class in src/heretic/scorer.py; get_score is the only required method. A plugin is referenced either as a fully qualified import path such as `fully.qualified.module.MyScorerClass` or as a file plus class name such as `path/to/plugin.py:MyScorerClass`, so custom scorers do not require editing the Heretic package.

ARA and abliteration sit inside an Optuna TPE loop

The other half of a run is the modifier, defined by the interface in src/heretic/modifier.py. Heretic wraps it in a TPE-based hyperparameter optimizer built on Optuna: the optimizer proposes parameters, the modifier applies them, and the scorers grade the resulting model. Two modifiers ship with the package. Arbitrary-Rank Ablation (ARA) is the default and was developed specifically for Heretic; its design is explained in pull request 211 and the code is in src/heretic/modifiers/ara.py. The second is directional ablation, the classic abliteration method for refusal suppression, in src/heretic/modifiers/abliteration.py. Heretic's version applies a flexible ablation weight kernel across layers and supports both projected abliteration and norm preservation (MPOA). None of this involves gradient training, which is why the tool runs on a single consumer GPU. The steering comes from contrasting example prompts and matching scorers, not from new training data.

Your own GPU sets the limit on which models you can modify

Heretic only modifies models that you can run on your own hardware. There is no hosted mode in the package, so a model that does not fit in your VRAM cannot be processed. The escape hatch is on-the-fly 4-bit quantization through bitsandbytes: set the `quantization` option to `bnb_4bit` and memory needs drop to roughly what a quantized copy of the model would use, at a slight accuracy cost. That cost lands in the resulting model, so you trade some quality for the ability to fit. Time is the other constraint. Heretic benchmarks the system at startup to pick a batch size, and with the default configuration an RTX 3090 needs about 30-40 minutes for Qwen3-4B-Instruct-2507. Coverage is broad but not universal: most dense models, many multimodal models, several MoE architectures and many hybrid models are supported, which leaves some architectures out. Check yours before planning around it.

PyTorch is left for you to install correctly

pyproject.toml pins most dependencies with compatible-release ranges, for example transformers[kernels]~=5.18, optuna~=5.0, bitsandbytes~=0.50, peft~=0.21 and lm-eval[hf]~=0.4. torch and torchvision are the exceptions: their versions are deliberately unspecified, and the usage instructions ask you to prepare PyTorch 2.6+ as appropriate for your hardware before installing heretic-llm. In practice this means the pip command does not settle which PyTorch build you end up with, and a build that does not match your GPU stack becomes your problem to sort out, not the package's. Supported Python versions run from 3.10 through 3.15 according to the package classifiers. The code is licensed AGPL-3.0-or-later, which matters if you plan to build a network service around a modified copy of the tool rather than just run it locally.

After the run: save, upload, chat or benchmark

When processing finishes, Heretic offers a set of actions: save the model locally, upload it to Hugging Face, chat with it to check how it behaves, run standard benchmarks on it, or any combination. The benchmark option is backed by the lm-eval dependency. Chatting is the quickest way to see whether refusals actually went away, while benchmarks show what the change cost on general tasks. The community has published thousands of models made with Heretic on Hugging Face, and the README points to independent comparisons on r/LocalLLaMA that measured Heretic outputs with MMLU and GSM8K and found them favorable against other abliteration tools. Those are third-party results, so run the benchmarks on your own output rather than assuming the same outcome for a different model or configuration.

Editorial conclusion

Heretic suits people who already run open-weight models locally and want refusal suppression, or a custom scored behavior change, without writing a training pipeline. It does not suit anyone whose model will not load on their GPU even with bnb_4bit quantization, or anyone who expects a 1.x config to keep working. Before a run, confirm your PyTorch 2.6+ build matches your hardware, check that your architecture is among the supported dense, MoE or hybrid families, and port old configs with docs/migration_v1_to_v2.md.

Frequently asked questions

Official sources

1. License: AGPL-3.0
2. p-e-w/heretic on GitHub
3. Project website
4. README
5. Releases