Remove refusals and adjust tone in local language models without retraining
You can remove refusal sentences or correct overly flattering tones in AI models running on your computer without needing a massive data center. Heretic is a tool that slightly adjusts the behavioral direction of a model using mathematical techniques, rather than retraining it. You can modify the model to your specifications within minutes, even without understanding its complex internal structure.
The principle of modifying models without training
Typically, changing the personality of an AI model requires retraining with substantial computational resources. Heretic skips this process and guides the model's output using comparison prompts and a score calculator. It works by optimizing metrics such as refusal frequency or the usage rate of specific words. This allows you to process 4B-scale models in 30 to 40 minutes using consumer-grade graphics cards like the RTX 3090. Once processing is complete, you can save the model, upload it to Hugging Face, or test it immediately in chat.
Specifying desired personalities via configuration files
The default configuration focuses on removing refusal responses. However, you can achieve different effects by changing the configuration file. Using config.noslop.toml reduces meaningless chatter, while config.nosyco.toml suppresses flattering tones. config.emotions.toml is used to adjust emotional alignment. Each configuration determines which scorer to activate and whether to minimize or maximize that score. You can use multiple scorers simultaneously or write code directly to create new evaluation criteria. Reduces meaningless chatter Suppresses flattering tones Adjusts emotional alignment
Reproducible model archive
Heretic records the model modification process, allowing you to recreate the exact same model byte-for-byte in the same environment. You can include this reproduction information when uploading models to Hugging Face. The reproduce.json file contains all the environment information necessary for model generation. Running the heretic --reproduce command checks the system environment and guides you to produce the same results as the original model. This process takes only a few minutes, rather than the several hours of computation required for the original. This collection of reproduction manifests is called the 'Heretic Grimoire,' and keeping a local backup allows you to restore the model at any time.
Pre-use considerations
You need an environment with Python 3.10 or higher and PyTorch 2.6 or higher. After installing with pip install -U heretic-llm, run it by specifying the name of the model to process. Configuration files from version 1.x are not compatible with version 2.x, so you must check the migration guide if you want to use previous configurations. It supports 4-bit quantization to reduce VRAM usage, though this may result in a slight loss of accuracy. This tool opens up model modification, previously possible only for large corporations, to individuals, but the legal and ethical responsibility for the resulting model lies with the user. It is distributed under the AGPL v3 license, and contributions must also follow the same license.
