Suggestions appear as you type. Use the up and down arrows to choose one and Enter to open it.

This page's audience our own tag's counts — open to see them
–Visits
–Page views
–Clicks to vendors
–Time on page
–Reading now
Clicks to vendors, by tool
  • –
Top countries
  • –
Devices
  • –

– · visits, page views and clicks counted by our own tag, which counts more than our checked analytics; every figure rounded down · how we count

Promptfoo review

Freemium#2 of 29 in LLM Evaluation ToolsAI Red Teaming ToolsAI LLM Evaluation ToolsAI Security Testing Tools+3 more

A broad evaluation toolkit for teams that need testing, security scans, and CI/CD gates.

9.3/10Editor score
Visit Promptfoo
Promptfoo9.3 Visit Promptfoo

Reviewed by iTechGuides Editors · Editorial team · Updated Oct 2026

Promptfoo is an open-source CLI and library for evaluating, testing, and red-teaming large-language-model applications. It is designed for developers and teams building AI applications that need to benchmark prompts, models, or RAG pipelines; compare outputs; define custom metrics; and review quality, safety, and security results. Its evaluation scope includes LLM-as-a-judge grading, human scoring, comments, labeling workflows, and automated vulnerability scanning.

The Community plan includes all LLM evaluation features, all model providers and integrations, red teaming, custom integration with your own app, local or self-hosted execution, and vulnerability scanning. It also includes up to 10,000 red-team probes per month at no charge. Enterprise adds team sharing, continuous monitoring, a centralized security and compliance dashboard, SSO, granular permission profiles, Promptfoo API access, and managed cloud deployment. On-Premise includes Enterprise features alongside deployment on your own infrastructure, complete data isolation, a dedicated runner, and an assigned deployment engineer. Enterprise and On-Premise pricing is custom, so the main buying distinction is between the free Community capabilities and managed collaboration, monitoring, governance, or deployment requirements.

Promptfoo fits teams that want one evaluation workflow spanning development, security testing, result comparison, and delivery controls. Integrations include OpenAI, Anthropic, Azure, Google, Hugging Face, Ollama, GitHub Actions, GitLab CI, Jenkins, Slack, Splunk, and MCP. Teams can run CI/CD quality gates and security scans, connect custom APIs and LLM providers, and use web-based result review. Choose Promptfoo when evaluation and red teaming need to work together across local, cloud, or self-hosted environments. Teams seeking only a narrow prompt comparison tool may prefer a more focused option.

Promptfoo pros and cons

  • Where it wins
    • Evaluates prompts, models, and RAG pipelines with custom assertions
    • Combines red teaming, vulnerability scanning, and safety evaluations
    • Supports local, self-hosted, cloud, API, and CI/CD workflows
  • Where it doesn't
    • Enterprise and On-Premise pricing is custom
    • Advanced collaboration and monitoring features require paid tiers
    • Its broad scope may exceed the needs of teams seeking only prompt comparison

Promptfoo fact sheet, pricing and score →

Advertiser disclosure: iTechGuides is reader-supported. Vendors can pay for top positions in our rankings and for a place on other products' pages, and we may earn a commission when you click some links. Bought places stand at the number bought; this page and our disclosure say so. Paying vendors may get a fuller, more detailed review from our editors, and that review can change their score. How we rank.

Last updated · How we research and update