Promptfoo
Promptfoo: A broad evaluation toolkit for teams that need testing, security scans, and CI/CD gates. Ranked #2 of 29 in LLM Evaluation Tools by our editors (9.3/10); pricing: Free plan; best for teams needing evaluation plus red teaming.
At a glance
- Editor score9.3 / 10
- PricingFree plan
- Best forTeams needing evaluation plus red teaming
- Free planYes
- Deployment optionsBoth
- Founded2024
- Facts checked20 Sep 2026
Where it wins
- Evaluates prompts, models, and RAG pipelines with custom assertions
- Combines red teaming, vulnerability scanning, and safety evaluations
- Supports local, self-hosted, cloud, API, and CI/CD workflows
Where it doesn't
- Enterprise and On-Premise pricing is custom
- Advanced collaboration and monitoring features require paid tiers
- Its broad scope may exceed the needs of teams seeking only prompt comparison
Our verdict on Promptfoo
Promptfoo is an open-source CLI and library for evaluating, testing, and red-teaming large-language-model applications. It is designed for developers and teams building AI applications that need to benchmark prompts, models, or RAG pipelines; compare outputs; define custom metrics; and review quality, safety, and security results. Its evaluation scope includes LLM-as-a-judge grading, human scoring, comments, labeling workflows, and automated vulnerability scanning.
The Community plan includes all LLM evaluation features, all model providers and integrations, red teaming, custom integration with your own app, local or self-hosted execution, and vulnerability scanning. It also includes up to 10,000 red-team probes per month at no charge. Enterprise adds team sharing, continuous monitoring, a centralized security and compliance dashboard, SSO, granular permission profiles, Promptfoo API access, and managed cloud deployment. On-Premise includes Enterprise features alongside deployment on your own infrastructure, complete data isolation, a dedicated runner, and an assigned deployment engineer. Enterprise and On-Premise pricing is custom, so the main buying distinction is between the free Community capabilities and managed collaboration, monitoring, governance, or deployment requirements.
Promptfoo fits teams that want one evaluation workflow spanning development, security testing, result comparison, and delivery controls. Integrations include OpenAI, Anthropic, Azure, Google, Hugging Face, Ollama, GitHub Actions, GitLab CI, Jenkins, Slack, Splunk, and MCP. Teams can run CI/CD quality gates and security scans, connect custom APIs and LLM providers, and use web-based result review. Choose Promptfoo when evaluation and red teaming need to work together across local, cloud, or self-hosted environments. Teams seeking only a narrow prompt comparison tool may prefer a more focused option.
Your product? Our editors can test Promptfoo in depth and publish a review, labelled as paid · from $149/mo →
Promptfoo pricing
All 3 Promptfoo plans and prices →
Promptfoo fact sheet
| Free plan | Yes |
|---|---|
| Paid from | Not verified |
| Deployment options | Both |
| Custom metrics | Yes |
| LLM-as-a-judge | Yes |
| Safety evaluations | Yes |
| Human review workflows | Yes |
| Prompt versioning | Not verified |
| CI/CD integration | Yes |
| Deployment | Cloud, Self-hosted |
| Platforms | Web |
| Compliance & security | SOC 2, ISO 27001 |
| Support | Community, Docs |
| Built for | Solo, Small business, Mid-market, Enterprise (editorial estimate) |
| Integrations | 12 integrations: OpenAI, Anthropic, Azure, Google, Hugging Face, Ollama … See all → |
| Pricing | Free plan (source) |
| Website | promptfoo.dev |
| Facts checked | 20 Sep 2026 |
Promptfoo integrations
Promptfoo lists 12 integrations on its own site.
- OpenAI
- Anthropic
- Azure
- Hugging Face
- Ollama
- GitHub Actions
- GitLab CI
- Jenkins
- Slack
- Splunk
- MCP
See all Promptfoo integrations →
Alternatives to Promptfoo
- DeepEvalA broad open-source toolkit for evaluating LLM apps, agents, RAG pipelines, and chatbots.9.4
- GalileoA mature evaluation platform spanning offline testing, production tracing, and runtime controls.9.1
- Maxim AIA broad evaluation workflow with a free plan and $29-per-seat Professional tier.9.0
See all Promptfoo alternatives →
Promptfoo vs the competition
- Promptfoo vs DeepEval
- Promptfoo vs Galileo
- Promptfoo vs Maxim AI
- Promptfoo vs Braintrust
- Promptfoo vs Confident AI
- Promptfoo vs Parea AI
Compare Promptfoo with any tool side by side →
Also Ranked In
Promptfoo is ranked in 9 of our directories; the 6 strongest besides LLM Evaluation Tools.
Used Promptfoo? Be the first to review it
The editor score above is our own research. What this page doesn't have yet is a reader's view — what you used Promptfoo for, what worked and what didn't. No stars are seeded and no review is paid for; an editor reads every one before it appears.
Write a reviewTwo minutes · verified accounts only · read by an editor before it appears
Featured on iTechGuides
Promptfoo is listed in our LLM Evaluation Tools directory. Add the badge to your site — it links back to this page.
<a href="https://www.itechguides.com/products/promptfoo/"><img src="https://www.itechguides.com/best/badge/promptfoo.svg" alt="Featured on iTechGuides" width="230" height="46"></a>
Reviewed by iTechGuides Editors · Editorial team · Updated Oct 2026
Advertiser disclosure: iTechGuides is reader-supported. Vendors can pay for top positions in our rankings and for a place on other products' pages, and we may earn a commission when you click some links. Bought places stand at the number bought; this page and our disclosure say so. Paying vendors may get a fuller, more detailed review from our editors, and that review can change their score. How we rank.
Last updated · How we research and update