No AI agent exceeds 12 percent on SemiAnalysis’ GDPx benchmark

SemiAnalysis has introduced GDPx, a benchmark that places AI agents in simulated professional environments rather than testing knowledge through prompts. The evaluation covers 206 tasks across 11 fields, including chip design, biology, quantitative trading, cybersecurity and law. Agents can work for up to eight hours with software such as Siemens NX, large data rooms and simulated enterprise systems. Administration and insurance are excluded.

Claude Opus 5.5 ranks first with a 12 percent score, followed by Gemini 4 Carbon at 10.9 percent and GPT-6 Astra at 9.6 percent. Gemini costs $16.29 per task, about one-third as much as Opus 5.5. GPT-6 Luna and DeepSeek 4.1 Flash score about five percent at roughly one dollar per task. Because a single spreadsheet error can invalidate a result, SemiAnalysis measures the proportion of individual requirements completed rather than fully solved tasks.

The results show weaknesses in several areas: 98 percent of 200 cyber runs violated at least one behavioral rule, and 70 of 111 quantitative-trading runs underperformed a reference strategy. Critics question the effect of safety filters, provider-specific agent software, hidden tasks and limited samples. SemiAnalysis plans updates every three to six months.