iTechGuides is reader-supported. When you buy through links on our site, we may earn an affiliate commission. As an Amazon Associate I earn from qualifying purchases. Learn more
Yes. AI agents have generated working smart-contract exploits in controlled benchmark environments, including tests against historical vulnerabilities and a smaller experiment that found two previously unknown flaws. Those demonstrations show real technical capability, but their reported dollar figures are simulated: the studies do not establish that the agents stole funds from live contracts.
What the results do—and do not—show
The headline finding is that an agent can sometimes turn a vulnerable contract into an executable exploit under test conditions. That is different from showing that AI can reliably find exploitable bugs in arbitrary deployed contracts, conduct attacks on live networks, or replace a security audit.
- Exploit generation: producing a transaction or script that succeeds against a vulnerable target in a controlled environment.
- Vulnerability detection: identifying and explaining a flaw. A benchmark may grade this separately from whether an exploit works.
- Patching: fixing the flaw while preserving the contract’s intended behavior. A successful exploit does not demonstrate a correct fix.
The distinction matters because a benchmark can demonstrate that a model performs a specific task on a selected test set without establishing the likelihood of success against an arbitrary contract in production.
Recommended Free Tools
What Anthropic’s SCONE-bench tested
Anthropic’s 2025 SCONE-bench report covers 405 contracts with real vulnerabilities exploited between 2020 and 2025 across Ethereum, Binance Smart Chain, and Base. In the historical tests, an agent receives a target contract in a Docker-contained setup with a local blockchain fork and tools exposed through MCP. The harness checks whether the agent’s final native-token balance exceeds a set threshold. The contracts were selected because they had already been exploited, so the results measure reproduction on known vulnerable targets—not the chance that a randomly chosen contract is vulnerable.
#1 Best Overall
- POWERFUL SECURITY KEY: The Security Key C NFC is the essential physical passkey for protecting your digital life from phishing attacks. It ensures only you can access your accounts.
- WORKS WITH 1000+ ACCOUNTS: Compatible with Google, Microsoft, and Apple. A single Security Key C NFC secures 100 of your favorite accounts, including email, password managers, and more.
- FAST & CONVENIENT LOGIN: Plug in your Security Key C NFC via USB-C and tap it, or tap it against your phone (NFC) to authenticate. No batteries, no internet connection, and no extra fees required.
- TRUSTED PASSKEY TECHNOLOGY: Uses the latest passkey standards (FIDO2/WebAuthn & FIDO U2F) but does not support One-Time Passwords. For complex needs, check out the YubiKey 5 Series.
- BUILT TO LAST: Made from tough, waterproof, and crush-resistant materials. Manufactured in Sweden and programmed in the USA with the highest security standards.
Historical exploit results
Across 10 evaluated models, Anthropic reported successful results on 207 of 405 benchmark problems (51.11%) under its Best@8 setup, with $550.1 million in simulated stolen funds. Best@8 is a benchmark evaluation setup; the figure is not a measure of live attack proceeds, expected profit, or the vulnerability rate of deployed contracts. The report estimates simulated returns using historical exchange rates and states that the tests ran in blockchain simulators rather than affecting real assets. Anthropic’s SCONE-bench report describes the test design and qualifications.
Anthropic also reported results on 19 problems it classified as post-knowledge-cutoff: 55.8% success and a maximum of $4.6 million in simulated stolen funds across the reported Opus 4.5, Sonnet 4.5, and GPT-5 results. These remain benchmark outcomes on selected, historically vulnerable cases—not observed thefts or a forecast of future losses.
Rank #2
- POWERFUL SECURITY KEY: The YubiKey 5C NFC is the most versatile physical passkey, protecting your digital life from phishing attacks. It ensures only you can access your accounts
- WORKS WITH 1000+ ACCOUNTS: Compatible with popular accounts like Google, Microsoft, and Apple. A single YubiKey 5C NFC secures 100+ of your favorite accounts, including email, password managers, and more
- FAST & CONVENIENT LOGIN: Plug in your YubiKey 5C NFC via USB and tap it, or tap it against your phone (NFC), to authenticate. No batteries, no internet connection, and no extra fees required
- MOST SECURE PASSKEY: Supports FIDO2/WebAuthn, FIDO U2F, Yubico OTP, OATH-TOTP/HOTP, Smart card (PIV), and OpenPGP. That means it’s versatile, working almost anywhere you need it
- PRIMARY & SPARE KEYS: Just like having a spare house key, we recommend buying two YubiKeys - one for daily use and one as a spare. That way you’ll never get locked out of your accounts
Separate experiment on recently deployed contracts
In a distinct experiment, Anthropic tested two agents against 2,849 recently deployed contracts without known vulnerabilities and reported finding two novel vulnerabilities. Their simulated exploit value was $3,694; the report says GPT-5 incurred $3,476 in API cost. This small proof of concept suggests agents may help surface previously unknown issues, but it does not show that the systems attacked live contracts or extracted real funds.
How EVMbench differs
OpenAI and Paradigm introduced EVMbench in 2026 with 117 curated vulnerabilities from 40 audits. The benchmark evaluates three distinct modes on contract-security tasks, primarily drawn from open code-audit competitions, with some scenarios from Tempo’s security-auditing process. Its task mix and grading differ from SCONE-bench, so the scores are not directly interchangeable.
Rank #3
- POWERFUL SECURITY KEY: The YubiKey 5 NFC is the most versatile physical passkey, protecting your digital life from phishing attacks. It ensures only you can access your accounts
- WORKS WITH 1000+ ACCOUNTS: Compatible with popular accounts like Google, Microsoft, and Apple. A single YubiKey 5 NFC secures 100+ of your favorite accounts, including email, password managers, and more
- FAST & CONVENIENT LOGIN: Plug in your YubiKey 5 NFC via USB and tap it, or tap it against your phone (NFC), to authenticate. No batteries, no internet connection, and no extra fees required
- MOST SECURE PASSKEY: Supports FIDO2/WebAuthn, FIDO U2F, Yubico OTP, OATH-TOTP/HOTP, Smart card (PIV), and OpenPGP. That means it’s versatile, working almost anywhere you need it
- PRIMARY & SPARE KEYS: Just like having a spare house key, we recommend buying two YubiKeys - one for daily use and one as a spare. That way you’ll never get locked out of your accounts
| Benchmark mode | What the agent must do | What is graded |
|---|---|---|
| Detect | Find vulnerabilities in a smart-contract repository | Recall of known issues; the authors note that additional findings can be difficult to adjudicate as genuine bugs or false positives |
| Patch | Change vulnerable contracts without breaking intended behavior | Automated tests and exploit checks; preserving functionality while removing exploitability remains challenging |
| Exploit | Execute an attack in a sandboxed blockchain environment | Automated transaction replay and verification of on-chain state |
In the comparison reported by OpenAI, GPT-5.3-Codex scored 71.0% in exploit mode and GPT-5 scored 33.3%. Those percentages belong to EVMbench’s tasks and grading setup, not to all contracts or real-world attacks. The EVMbench introduction explains the benchmark’s three modes and reported comparison; the paper record in Proceedings of Machine Learning Research describes its dataset and evaluation.
Why benchmark scores do not predict live attacks
Selection and coverage
SCONE-bench’s historical cases are known to have been exploited, while EVMbench’s cases are curated from audits. Both designs make repeatable testing possible, but neither provides a representative sample from which to calculate the share of all production contracts that an agent could exploit. OpenAI also cautions that some widely deployed, heavily scrutinized contracts may be harder to exploit than benchmark tasks.
Rank #4
- POWERFUL SECURITY KEY: The Security Key NFC is the essential physical passkey for protecting your digital life from phishing attacks. It ensures only you can access your accounts.
- WORKS WITH 1000+ ACCOUNTS: Compatible with Google, Microsoft, and Apple. A single Security Key NFC secures 100 of your favorite accounts, including email, password managers, and more.
- FAST & CONVENIENT LOGIN: Plug in your Security Key NFC via USB-A and tap it, or tap it against your phone (NFC) to authenticate. No batteries, no internet connection, and no extra fees required.
- TRUSTED PASSKEY TECHNOLOGY: Uses the latest passkey standards (FIDO2/WebAuthn & FIDO U2F) but does not support One-Time Passwords. For complex needs, check out the YubiKey 5 Series.
- BUILT TO LAST: Made from tough, waterproof, and crush-resistant materials. Manufactured in Sweden and programmed in the USA with the highest security standards.
Harness and grading choices
EVMbench’s authors describe limits that include uncertain adjudication of extra findings, sequential replay, lack of precise timing mechanics, local rather than mainnet-forked state, and support for a single chain. Such choices affect what an agent can demonstrate and how a score should be interpreted. A high result under one harness does not automatically transfer to different networks, contract states, timing conditions, or defensive controls.
Crashes, No Sound, or Screen Glitches?
Random freezes, missing sound and display glitches usually trace back to one bad driver. Find and replace yours safely.Free scan · under a minuteWindows Errors? Fix Them Before They Spread
Repair common Windows errors and clear accumulated junk for a smoother, more stable PC - no reinstall needed.Free scan · no reinstallScores change with the evaluation
Results depend on the model version, prompt, number of trials, benchmark version, and grading design. Anthropic’s later SCONE-bench update illustrates that reported comparisons continue to change. Treat every percentage as a dated result on a named benchmark rather than a stable ranking of general security ability.
Best Value
- POWERFUL SECURITY KEY: The YubiKey 5 is a versatile physical passkey that protects your digital life from phishing attacks. It ensures only you can access your accounts.
- WORKS WITH 1000+ ACCOUNTS: Compatible with popular accounts like Google, Microsoft, and Apple. A single YubiKey 5 secures 100+ of your favorite accounts, including email, password managers, and more.
- FAST & CONVENIENT LOGIN: Plug in your YubiKey 5 via USB and tap it to authenticate. No batteries, no internet connection, and no extra fees required.
- MOST SECURE PASSKEY: Supports FIDO2/WebAuthn, FIDO U2F, Yubico OTP, OATH-TOTP/HOTP, Smart card (PIV), and OpenPGP. That means it’s versatile, working almost anywhere you need it.
- BUILT TO LAST: Made from tough, waterproof, and crush-resistant materials. Manufactured in Sweden and programmed in the USA with the highest security standards.
How development teams can use agent-generated exploits
For a contract team, exploit generation is best treated as one input to authorized security testing—not as proof that a system is safe or unsafe. The benchmark work supports exploration of pre-deployment testing and defensive patching, but it does not establish that AI-only review provides complete assurance.
- Keep testing isolated and authorized. Use local chains, simulators, or other environments where the team controls the target and assets.
- Require a reproducible proof of concept. Confirm what state changes, which assumptions are required, and whether the result depends on the test harness.
- Review the finding and the proposed fix. A human reviewer should validate that the issue is real and that a patch removes the exploit without changing intended behavior.
- Retest independently. Run the relevant tests and exploit checks after the fix, and retain independent audit and deployment controls.
Do not treat simulated balances as losses, use an agent’s success on a benchmark as permission to target third-party contracts, or infer safety from an agent failing to find a bug. The studies demonstrate a capability under defined conditions; they do not settle the security of any particular live contract.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
