TrustLLM: running the ICML 2024 trustworthiness benchmark on local models

Six dimensions, and a benchmark you can run yourself

TrustLLM is an open research toolkit for evaluating the trustworthiness of large language models across six dimensions, and it is the code that goes with the ICML 2024 paper, with the paper on arXiv, a documentation site, a dataset on Hugging Face and a public leaderboard all linked from the documentation. The six dimensions are the backbone of the project, and the table in the documentation deserves a close read. Truthfulness covers misinformation, hallucination and sycophancy. Safety covers jailbreaks, misuse and exaggerated safety. Fairness covers stereotypes, preferences and disparagement. Robustness covers adversarial perturbations and out-of-domain inputs. Privacy covers privacy awareness and information leakage, and ethics covers moral judgments and moral choices. The audience is anyone who has to make a claim about how a model behaves, since the value of the toolkit is that the evaluation is a published benchmark with a dataset and a leaderboard rather than a set of ad hoc prompts.

Installation is a git URL, and reproducibility means pinning a SHA

The base package supports API generation and data downloads, and it does not require GPU libraries. The install command points at GitHub rather than a package index:

python -m pip install "trustllm @ git+https://github.com/HowieHwong/TrustLLM.git@main#subdirectory=trustllm_pkg"

Two details in that line matter. The subdirectory fragment points the installer at the trustllm_pkg directory inside the repository, which is where the package actually lives. And the version reference is main, which the documentation explicitly frames as installing the 0.4 source version directly from GitHub, with Git required. For reproducible runs the documentation says to replace main with a commit SHA, and for a benchmark whose output you may want to cite later, that is the difference between a result you can repeat and one you cannot. The repository layout backs this up, with the package in trustllm_pkg/ and everything else, the dataset/, docs/, examples/, scripts/, tests/, hooks/, images/ and the documentation configuration with mkdocs.yml, living outside it.

Download the data, then list what the suite contains

Before any model runs, the benchmark data has to be on disk, and the CLI has a dedicated command for that:

python -m trustllm download --output data
python -m trustllm tasks

The download command writes into a directory you name, here data, and the tasks command lists the evaluation tasks that are available. The separation between commands is the structural point of the 0.4 release, which lists download, tasks, generate and evaluate as the command set, all also reachable through python -m trustllm. Generation and scoring are distinct stages with distinct outputs, so a team can run generation once against several models and score the resulting response files later, and the documentation says JSON response files remain compatible with the existing res-based evaluators. That last point is what keeps old experiment directories usable.

API mode talks to any OpenAI-compatible endpoint

The API path is the one most teams will try first, because it needs no weights on the machine. Credentials and endpoint come from two environment variables:

export OPENAI_API_KEY="your-api-key"
export OPENAI_BASE_URL="https://your-provider.example/v1"

python -m trustllm generate \
  --backend api --model your-model-id \
  --task safety --data data/dataset \
  --limit 5 --concurrency 4 --output runs/api-safety

The model identifier is arbitrary, with no legacy whitelist in 0.4, so the same command covers a hosted provider, a local OpenAI-compatible server on http://localhost:8000/v1 or a self hosted gateway, and an API key is optional when the server does not require one. API mode uses text-only Chat Completions, which is the constraint to keep in mind when comparing it with local mode. The limit flag deserves attention: the documentation says it selects five samples per dataset file for a smoke test, so a full run means removing it and choosing a new output directory. The run writes a report.html with completion counts, and the documentation is careful to say that this report tracks generation, not benchmark scores.

Local mode runs the same workflow over Hugging Face weights

The local path installs an extra dependency group and changes one flag. The extra is called local:

python -m pip install "trustllm[local] @ git+https://github.com/HowieHwong/TrustLLM.git@main#subdirectory=trustllm_pkg"

python -m trustllm generate \
  --backend local --model Qwen/Qwen2.5-0.5B-Instruct \
  --task safety --data data/dataset \
  --device auto --limit 5 --output runs/local-safety

The example model is a small instruct-tuned Qwen checkpoint, which is the sensible choice for a smoke test. Weights download on first use, and the documentation says to replace the model ID with a path to a checkpoint to use weights that are already on the machine. Device selection is explicit, with cpu, cuda:0 and mps available, and auto using Accelerate placement. Local loading supports Transformers causal language models and uses the tokenizer's chat template when one is available, which is why the format can differ from the API path. The documentation also keeps the limits visible: model access, hardware capacity and architecture compatibility still apply.

Scoring is a separate stage that can call a judge model

Scores come from a second dependency group and a second command:

python -m pip install "trustllm[eval] @ git+https://github.com/HowieHwong/TrustLLM.git@main#subdirectory=trustllm_pkg"
python -m trustllm evaluate --task safety --data runs/api-safety-full

The path has to be a full run output directory, and the results land in scores.json and scores.html, with incomplete responses rejected rather than silently scored. The six scoring pipelines retain the original benchmark methods, but what that means in practice varies by task: depending on the task, scoring uses rules, a downloaded classifier, embeddings, or an API judge. The judge path is configured with the OPENAI_JUDGE_MODEL variable, and the documentation states plainly that judge calls can incur charges. The environment example lists a long tail of provider keys, including Perspective, Anthropic, DeepInfra, Replicate, Gemini, Zhipu, Ernie and a set of Azure OpenAI variables, which shows how many endpoints the scoring code can reach for.

The 0.4 rewrite changed the generation path, and the docs admit it

The changelog entry in the documentation is the part to read before comparing numbers with the paper. The changes are listed: arbitrary model IDs and a shared runner for API and local generation with no legacy model whitelist, a lightweight API install with optional local, eval and archived legacy dependency groups, the four commands already described, and operational details such as bounded API retries, per-sample checkpoints, explicit failures and a guarded resume flag. Runs also record dataset hashes, settings, dependency versions, usage where returned and HTML generation reports, which is the kind of provenance a benchmark result should carry. The uncomfortable sentence is that the original generation engine is archived under trustllm.generation.legacy, and that generation formatting and model integrations have changed, so new runs are not automatically equivalent to the original paper settings. The migration notes point is where to look for the details. The releases are v0.4.0 on 2026-10-04 and v0.3.0 on 2024-04-23, with the last push to the repository on 2026-10-04, and the project also publishes a machine-readable documentation index for agent tooling plus a dedicated agent integration guide.

Editorial conclusion

TrustLLM suits research and platform teams that need a published, reproducible trustworthiness benchmark rather than one written from scratch, since the six dimensions and the original scoring methods are kept. It is the wrong tool if you need the numbers to match the ICML 2024 paper exactly, because the migration notes state that new runs are not automatically equivalent to the original settings. Before starting, read the scoring guide, because some tasks need a downloaded classifier, embeddings or an API judge, and judge calls can incur charges, and pin a commit SHA rather than tracking main for a run you intend to cite.

Frequently asked questions

Official sources

1. HowieHwong/TrustLLM on GitHub
2. License: MIT
3. Project website
4. README
5. Releases