Suggestions appear as you type. Use the up and down arrows to choose one and Enter to open it.

τ-bench

Free#33 of 39 in AI Model Evaluation Software

τ-bench: A focused, self-hosted benchmark for tool-using agent conversations. Ranked #33 of 39 in AI Model Evaluation Software by our editors (5.8/10); pricing: Free plan; best for tool-using agent conversation benchmarks.

5.8/10Editor score
τ-bench5.8 Visit τ-bench

At a glance

  • Editor score
    5.8 / 10
  • Pricing
    Free plan
  • Best for
    Tool-using agent conversation benchmarks
  • Free plan
    Yes
  • Paid from
    Free
  • Safety & bias checks
    No
  • Founded
    2024
  • Facts checked
    19 Nov 2026
  • Where it wins

    • Simulates dynamic conversations between users and language agents
    • Evaluates airline and retail API-tool interactions against policies
    • Reports pass@k leaderboard metrics across configurable agent strategies
  • Where it doesn't

    • Scope centers on airline and retail environments
    • Self-hosted deployment requires setup and model-provider API keys
    • Repository tasks are outdated; newer domains require τ³-bench

Our verdict on τ-bench

τ-bench is an open-source benchmark for evaluating language-model agents that converse with simulated users and call domain-specific API tools under policy guidelines. It is aimed at research and engineering teams studying conversational task success, tool adherence and agent behavior across small, mid-market and enterprise settings. The project is distributed as installable Python source and runs through self-hosted deployment.

Its strongest area is controlled interaction evaluation. Teams can configure model and user simulators, select task IDs, control concurrency and inspect historical trajectory files. Airline and retail environments provide concrete API-tool scenarios, while Act, ReAct and tool-calling strategies support different agent patterns. Policy-guideline scoring adds an adherence dimension, and pass@k leaderboard metrics give a consistent way to compare repeated task outcomes. The category specifications list model-provider support for OpenAI, Anthropic, Google, Mistral and AnyScale.

The published project is free and open source, but operating it means managing a self-hosted installation and supplying model-provider API keys. Community support is available rather than 24/7 coverage. τ-bench is a sensible choice when your evaluation program specifically needs simulated user-agent conversations with airline or retail tools, configurable runs and regression benchmarking. It is less suitable for safety-bias testing, human review workflows or evaluations outside its supplied interaction patterns. The maintainers state that the airline and retail tasks are outdated and direct users to τ³-bench for newer domains and fixed tasks, so teams needing current scenarios should consider that project instead.

Your product? Our editors can test τ-bench in depth and publish a review, labelled as paid · from $149/mo →

τ-bench pricing

Plans Free planFree Free to use — no paid tier required for the core job.
See plans on github.com

τ-bench fact sheet

Free planYes
Paid fromFree
Evaluation typestool-agent-user interaction; conversational task success; policy and API-tool adherence
Safety & bias checksNo
Regression benchmarksYes
Human review workflowsNo
Included evaluation runsNot verified
Deployment optionSelf_hosted
Model providersOpenAI; Anthropic; Google; Mistral; AnyScale
DeploymentSelf-hosted
SupportCommunity
Built forSmall business, Mid-market, Enterprise (editorial estimate)
PricingFree plan
Websitegithub.com
Facts checked19 Nov 2026

Alternatives to τ-bench

See all τ-bench alternatives →

Used τ-bench? Be the first to review it

The editor score above is our own research. What this page doesn't have yet is a reader's view — what you used τ-bench for, what worked and what didn't. No stars are seeded and no review is paid for; an editor reads every one before it appears.

Write a reviewTwo minutes · verified accounts only · read by an editor before it appears

Reviews come only from verified accounts. Sign in or create an account first — your e-mail is never shown.

Your rating

0 characters · at least 80, up to 3,000

Posted from your verified account. Reviews appear after an editor reads them, usually within two working days.

Featured on iTechGuides

Featured on iTechGuides — τ-bench 5.8/10

τ-bench is listed in our AI Model Evaluation Software directory. Add the badge to your site — it links back to this page.

<a href="https://www.itechguides.com/products/bench-ai-model-evaluation-tool/"><img src="https://www.itechguides.com/best/badge/bench-ai-model-evaluation-tool.svg" alt="Featured on iTechGuides" width="230" height="46"></a>

Reviewed by iTechGuides Editors · Editorial team · Updated Nov 2026

Advertiser disclosure: iTechGuides is reader-supported. Vendors can pay for top positions in our rankings and for a place on other products' pages, and we may earn a commission when you click some links. Bought places stand at the number bought; this page and our disclosure say so. Paying vendors may get a fuller, more detailed review from our editors, and that review can change their score. How we rank.

Last updated · How we research and update