Index v5: open weights trail the leader by 26 points, not 12

Felix Laurent 29 Oct 2026 4 min read

models

Artificial Analysis says its new Intelligence Index v5 widens the open weights gap from 3 points to 12. That figure rests on a model nobody can download yet. Among released weights, the gap to the leader is 26 points.

The benchmark publisher announced the update on Oct 27 and promoted it on X a day later. It features 2 new benchmarks and 3 upgrades. It also says 60% of the index is now based on evaluations with private test sets or held-out answers.

What changed in index v5

The two new benchmarks are FrontierCode 2.0 and Terminal-Bench-Science. The three upgrades are HLE-Diamond, AA-Omniscience v1.1 and CritPt v2. Artificial Analysis also removed GDPval-AA v2.1, SciCode and AA-LCR v1.1.

Four benchmarks stay: AA-Briefcase, AutomationBench-AA, GDP.pdf and Terminal-Bench 4.0. According to the publisher, FrontierCode 2.0 launched a day earlier and has 132 tasks in real open-source repositories. Each task is graded on whether a maintainer would merge the agent's change.

Artificial Analysis launched a second product the same day. Progress Index v1 tracks model capability over time on one continuous scale. The new benchmarks also change how the score is built.

How the score is weighted

The index now has four equal parts of 25% each: knowledge work, coding, science and general. Within them, AA-Briefcase, FrontierCode 2.0, Terminal-Bench-Science and AA-Omniscience v1.1 carry 15% each.

HLE-Diamond is a 1,000-question version of Humanity's Last Exam, split evenly between reasoning and knowledge. CritPt v2 keeps only the 40 hardest of its 70 questions, dropping the 30 that seven frontier models solved every time.

Agentic evaluations make up 55% of the weight, the publisher says. They also use a median of about 90% of all output tokens in a run. That shift toward long agent tasks shapes who comes out on top.

Artificial Analysis Intelligence Index v5: Gemini 4 Carbon leads at 59, Claude Opus 5.5 58, GPT-6 Astra 57; open weights Kimi K3.1 Pro 46, DeepSeek V4.1 Flash and GLM-5.3 at 32

Who leads the index

At launch, Claude Opus 5.5 (max) led at 58, according to Artificial Analysis. GPT-6 Astra (max) followed at 57 and GPT-6.1 Sol (max) at 55. Anthropic and OpenAI held the top five places. Gemini 4 Argon (high) was next at 50.

A day later the top changed. Google released Gemini 4 Carbon on Oct 28, and Artificial Analysis lists it at 59, #1 on the leaderboard. That pushes every gap in this article up by one point. The gap that matters most is the one for open weights.

The open weights gap

The publisher says the open weights gap widens from 3 points to 12. The leading open weights model is Kimi K3.1 Pro at 46. Its weights are not yet released.

Moonshot AI released the model on Oct 23. A report by R&D World, retelling the company's statement, says Moonshot will publish the full weights by Nov 4. As of Oct 27, there was still no K3.1 repository on Moonshot's Hugging Face page, according to later reports. Until then, the 46 is not a score anyone can reproduce with downloaded files.

Among released weights, DeepSeek V4.1 Flash 1017 (max) and GLM-5.3 (max) lead at 32. That is 26 points behind the launch leader, and 27 behind Carbon. It is also below Claude Haiku 5.5 (max), which scores 36. By this measure, the gap is more than double the headline one.

The publisher's earlier launch article for Kimi K3.1 Pro put the numbers the other way round. It said that once the Kimi weights are released, the gap between the leading open weights and proprietary models would narrow from 12 points to 3. The v5 post says the gap widens from 3 to 12. A score also has a price.

Cost per task

Cost varies widely among the top models. Of the 10 highest-scoring models shown, GPT-6.1 Terra (max) is the cheapest at $0.96 per index task. Claude Fable 5.1 (max) is the most expensive at $13.04. Claude Opus 5.5 (max) costs $10.04, GPT-6 Astra (max) $6.24 and GPT-6.1 Sol (max) $1.27.

The exact ratio of the two extremes is about 13.6. The comparison covers only the top 10 models shown, not the whole index. Effort settings matter as much as the model. Opus 5.5 at low effort costs $0.98 per task but scores 35 instead of 58.

Artificial Analysis Intelligence Index v5 cost per task: Claude Fable 5.1 $13.04, Claude Opus 5.5 $10.04, GPT-6 Astra $6.24, GPT-6.1 Sol $1.27, GPT-6.1 Terra $0.96

What cannot be checked

Every number here comes from one post by the benchmark's own publisher. A report by Trending Topics notes that because most test data is now secret, outsiders can no longer verify results independently. FrontierCode 2.0 is also new, and its private datasets are not open to inspection. Artificial Analysis does publish a public subset of AA-Omniscience v1.1 on Hugging Face.

So how far behind are open weights? Further than the headline says. The 12-point figure depends on Kimi K3.1 Pro, whose weights are promised but not out. Among models anyone can download today, the gap to the leader is 26 points at launch and 27 since Carbon. Even a closed small model, Claude Haiku 5.5, outscores them. If Moonshot ships the weights by Nov 4, the real gap shrinks to about 13 points. Until then, the honest number is the bigger one.