Artificial Analysis overhauls its Intelligence Index after GPT-6 Astra scoring drew skepticism

Artificial Analysis has released version 4.2 of its Intelligence Index, likely in response to criticism that its benchmarks failed to capture GPT-6 Astra's actual progress.

Earlier, several evaluations, including OpenAI's own, had placed Astra well ahead of the field. Epoch AI ranked it first out of 267 models with 169 points across more than 50 benchmarks, and ARC-AGI-3 showed a large to very large jump depending on the harness used. Artificial Analysis scored Astra just on par with its predecessor.

With the updated index, GPT-6 Astra now shows a four-point gain over its predecessor. Anthropic's Claude Fable 5.1 still leads the ranking, followed by Astra in second and Meta in third. Astra also uses fewer tokens per task than every other frontier model, according to Artificial Analysis. On cost-to-performance ratio, Anthropic, OpenAI, Meta, and Zhipu AI all share the lead.

The index adds two new benchmarks: AA-Briefcase for real-world knowledge work and GDP.pdf from Surge AI for PDF document analysis. GPQA-Diamond has been dropped because models have solved it. Private test data now makes up 40 percent of the weighting to make gaming harder. Artificial Analysis also fixed scoring errors across several benchmarks and tweaked its grading systems for more stable results.

Artificial Analysis says it held off on updates to keep scores stable during major model launches, but the top of the leaderboard moved so fast that an interim update was necessary. Version 5 has been in the works for eight months and will roll out in stages, it says.

AI News Without the Hype – Curated by Humans

Subscribe to THE DECODER for ad-free reading, a weekly AI newsletter, our exclusive "AI Radar" frontier report six times a year, full archive access, and access to our comment section.