DecodingTrust
DecodingTrust: A safety-focused research toolkit with broad evaluations and self-hosted execution. Ranked #27 of 29 in LLM Evaluation Tools by our editors (6.6/10); pricing: Open source; best for safety researchers studying trust and robustness.
At a glance
- Editor score6.6 / 10
- PricingOpen source
- Best forSafety researchers studying trust and robustness
- Paid fromFree
- Deployment optionsSelf-hosted
- Facts checked20 Sep 2026
Where it wins
- Covers toxicity, bias, robustness, privacy, ethics, and fairness
- Supports configurable model and dataset evaluation runs
- Runs through Docker, Singularity, Python packages, or Git
Where it doesn't
- Requires self-hosted deployment
- Does not provide verified custom metrics
- Lacks integrations and CI support
Our verdict on DecodingTrust
DecodingTrust is an open-source research benchmark and evaluation toolkit for assessing the trustworthiness of large language models. It is designed for safety researchers and practitioners who need structured evaluations across toxicity, stereotype and bias, adversarial robustness, out-of-distribution robustness, privacy, adversarial demonstrations, machine ethics, and fairness. The project provides datasets, evaluation scripts, configuration files, and reproducible workflows. It primarily evaluates GPT-3.5 and GPT-4, while also supporting Hugging Face and locally hosted causal language models.
Its main strength is evaluation breadth within safety and trustworthiness research. Configurable model and dataset evaluation runs let teams shape experiments around their research questions instead of relying on a single fixed benchmark. Docker and Singularity execution support reproducible local workflows, while installation is available through Python packages, Git, Docker, or Singularity. This makes DecodingTrust a fit for teams that can manage their own environment and want direct control over datasets, scripts, and model evaluation.
The trade-off is its self-hosted, research-oriented structure. DecodingTrust is not presented as a hosted commercial SaaS product, and its positioning does not include verified custom metrics, integrations, or CI support. Teams seeking a focused safety benchmark with local execution should consider it; teams that need managed infrastructure, connected evaluation workflows, or continuous integration support should choose an alternative. Its scope is strongest when trust, robustness, privacy, and fairness are central requirements rather than secondary checks.
Your product? Our editors can test DecodingTrust in depth and publish a review, labelled as paid · from $149/mo →
DecodingTrust pricing
DecodingTrust fact sheet
| Free plan | Not verified |
|---|---|
| Paid from | Free |
| Deployment options | Self-hosted |
| Custom metrics | Not verified |
| LLM-as-a-judge | Not verified |
| Safety evaluations | Yes |
| Human review workflows | Not verified |
| Prompt versioning | Not verified |
| CI/CD integration | Not verified |
| Deployment | Self-hosted |
| Support | Email, Docs |
| Pricing | Open source |
| Website | decodingtrust.github.io |
| Facts checked | 20 Sep 2026 |
Alternatives to DecodingTrust
- DeepEvalA broad open-source toolkit for evaluating LLM apps, agents, RAG pipelines, and chatbots.9.4
- PromptfooA broad evaluation toolkit for teams that need testing, security scans, and CI/CD gates.9.3
- GalileoA mature evaluation platform spanning offline testing, production tracing, and runtime controls.9.1
See all DecodingTrust alternatives →
Used DecodingTrust? Be the first to review it
The editor score above is our own research. What this page doesn't have yet is a reader's view — what you used DecodingTrust for, what worked and what didn't. No stars are seeded and no review is paid for; an editor reads every one before it appears.
Write a reviewTwo minutes · verified accounts only · read by an editor before it appears
Featured on iTechGuides
DecodingTrust is listed in our LLM Evaluation Tools directory. Add the badge to your site — it links back to this page.
<a href="https://www.itechguides.com/products/decodingtrust/"><img src="https://www.itechguides.com/best/badge/decodingtrust.svg" alt="Featured on iTechGuides" width="230" height="46"></a>
Reviewed by iTechGuides Editors · Editorial team · Updated Oct 2026
Advertiser disclosure: iTechGuides is reader-supported. Vendors can pay for top positions in our rankings and for a place on other products' pages, and we may earn a commission when you click some links. Bought places stand at the number bought; this page and our disclosure say so. Paying vendors may get a fuller, more detailed review from our editors, and that review can change their score. How we rank.
Last updated · How we research and update