garak

At a glance

• Editor score
  7.8 / 10
• Pricing
  Free plan
• Best for
  Security teams testing LLM attack resistance
• Free plan
  Yes
• Paid from
  Free
• Deployment options
  Self-hosted
• Facts checked
  20 Sep 2026

• Where it wins

  • Tests prompt injection, jailbreaks, toxicity, and other failure modes
  • Supports hosted and local models through multiple interfaces
  • Produces structured JSONL run reports and hit logs

• Where it doesn't

  • Focuses on adversarial testing rather than general evaluation coverage
  • Self-hosted deployment places setup and maintenance on the team
  • Command-line workflows may require technical implementation effort

Our verdict on garak

garak is a free, open-source command-line toolkit for assessing the security and robustness of large language models and language-model systems. It runs adversarial probes against models or endpoints, applies detectors to generated outputs, and produces structured reports. Its coverage includes prompt injection, jailbreaks, hallucination, data leakage, misinformation, toxicity, and malicious-code generation. Security teams, researchers, and developers evaluating LLM applications are the clearest audience, particularly when testing attack resistance is more important than broad model benchmarking.

The main strength is its focused safety workflow. Built-in detectors identify specific failure modes, including LLM-as-a-judge detectors, while extensible probes, detectors, evaluators, and generators allow teams to expand assessments. Support for hosted and local model interfaces includes Hugging Face, Replicate, the OpenAI API, AWS Bedrock, LiteLLM, llama.cpp, REST endpoints, and NVIDIA NIM. That combination fits environments where models may run through different providers or on local infrastructure, and the JSONL run reports and hit logs give teams structured outputs for review and follow-up.

garak is distributed as free open-source software and uses a self-hosted deployment model. The published installation materials cover Windows, macOS, and Linux, with API-based model access also supported. Its narrower scope is important: garak is designed as a specialist for adversarial probes, safety testing, and unwanted behaviors rather than a general evaluation suite. Choose it when LLM security testing, jailbreak resistance, prompt-injection analysis, or toxicity checks are central requirements. Teams seeking wider evaluation coverage beyond these safety and robustness concerns should consider a broader evaluation tool instead.

Your product? Our editors can test garak in depth and publish a review, labelled as paid · from $149/mo →

garak pricing

garak fact sheet

garak integrations

garak lists 8 integrations on its own site.

• Hugging Face
• Replicate
• OpenAI API
• AWS Bedrock
• LiteLLM
• llama.cpp
• REST endpoints
• NVIDIA NIM

Alternatives to garak

• DeepEvalA broad open-source toolkit for evaluating LLM apps, agents, RAG pipelines, and chatbots.9.4
• PromptfooA broad evaluation toolkit for teams that need testing, security scans, and CI/CD gates.9.3
• GalileoA mature evaluation platform spanning offline testing, production tracing, and runtime controls.9.1

See all garak alternatives →

Featured on iTechGuides

garak is listed in our LLM Evaluation Tools directory. Add the badge to your site — it links back to this page.

<a href="https://www.itechguides.com/products/garak/"><img src="https://www.itechguides.com/best/badge/garak.svg" alt="Featured on iTechGuides" width="230" height="46"></a>

Reviewed by iTechGuides Editors · Editorial team · Updated Oct 2026

Advertiser disclosure: iTechGuides is reader-supported. Vendors can pay for top positions in our rankings and for a place on other products' pages, and we may earn a commission when you click some links. Bought places stand at the number bought; this page and our disclosure say so. Paying vendors may get a fuller, more detailed review from our editors, and that review can change their score. How we rank.

Last updated 7 Oct 2026 · How we research and update