CTFBench - Benchmarking AI Smart Contract Auditors
AI summary of the linked articleCTFBench is a benchmark for evaluating AI smart contract auditors, using test contracts that each contain exactly one known vulnerability. It scores auditors on a Vulnerability Detection Rate (the proportion of known vulnerabilities identified) and an Overreporting Index (false positives per line of code in contracts known to be free of vulnerabilities). In the listed results, SavantChat Dec 2025 scores 1.000 VDR with an OI of 0.005, while gpt_5.5 also scores 1.000 VDR but with an OI of 0.098. The page's table also lists gemini_3.1_pro at 0.984 VDR and claude_opus_4.7 at 0.730 VDR with an OI of 0.202. |
Comments No comments yetWhat caught your eye? Start the discussion. |
