AgentDojo review
A self-hosted Python framework for benchmarking prompt injection and agent defenses.
Reviewed by iTechGuides Editors · Editorial team · Updated Oct 2026
AgentDojo is an open-source Python framework and benchmark for evaluating prompt-injection attacks and defenses in tool-using LLM agents. It is designed for researchers and developers who need to run repeatable security evaluations across realistic workflows. The project includes task suites, simulated environments, attack implementations, defense components, model pipelines, and measurements for both security and utility. Researchers benchmarking agent defenses will find its focus especially relevant.
Its strongest advantage is the breadth of evaluation building blocks available within a free, open-source project. AgentDojo includes prompt-injection attack benchmarking, built-in jailbreak attack classes, custom attack creation and registration, task authoring, and task-suite support. It also supports agent pipeline and defense composition, a prompt-injection detection component, and multiple LLM pipeline integrations. That combination supports comparisons between attacks, defenses, agent configurations, and task outcomes rather than limiting evaluation to a single attack type.
The main trade-off is deployment and scope. AgentDojo is self-hosted, so teams need to manage the environment in which the Python framework, benchmarks, and integrations run. Its purpose is also specific: evaluating attacks and defenses in tool-using LLM agents. The published capabilities do not extend to verifying leakage or unsafe-output testing. Choose AgentDojo when open, customizable benchmarking and security-and-utility evaluation are the priority. Consider an alternative when you need managed hosting or broader coverage of leakage and unsafe-output testing.
AgentDojo pros and cons
- Where it wins
- Free open source with prompt-injection and jailbreak benchmarks
- Create custom attacks, tasks, suites, pipelines, and defenses
- Measures both security and utility across agent workflows
- Where it doesn't
- Self-hosted deployment requires a suitable local environment
- Focused on tool-using LLM agent evaluation
- Does not verify leakage or unsafe-output testing
AgentDojo fact sheet, pricing and score →
Advertiser disclosure: iTechGuides is reader-supported. Vendors can pay for top positions in our rankings and for a place on other products' pages, and we may earn a commission when you click some links. Bought places stand at the number bought; this page and our disclosure say so. Paying vendors may get a fuller, more detailed review from our editors, and that review can change their score. How we rank.
Last updated · How we research and update