FrontierCode 2.0: Opus 5.5 tops at 45.5%, down from 54.6%

Felix Laurent 27 Oct 2026 4 min read

models

Cognition published the methodology, scoring criteria and sample tasks of FrontierCode 2.0 on its blog on Oct 27. The company says its top-scoring model, Claude Opus 5.5, reaches 45.5%. The same model scored 54.6% on FrontierCode 1.1.

What Cognition published

Cognition announced the benchmark on X on Oct 27 and pointed readers to its blog for the rules. The blog carries the scoring criteria and sample tasks. All scores in it are Cognition's own results, so the method matters more than the headline.

How the score works

The score is a weighted aggregate of rubric items. Solutions that do not pass blocking criteria receive 0. A single failed blocking item therefore removes the whole result for that task.

Cognition says it loosened its scope criteria in version 2.0. The aim is to accept surprising but acceptable behavior. According to the blog, the average scope failure rate fell from 9.0% on 1.1 to 3.8% on 2.0.

Bigger tasks than version 1.1

Cognition says reference solutions in 2.0 modify 67% more files and 89% more lines of code than in 1.1. The codebases are 25% larger. In 80% of tasks the reference solution differs from the upstream pull request, up from 60% in FrontierCode 1.0.

The task set also grew wider. Cognition reports 55% more repositories than in FrontierCode 1.0 and 43% more blocking criteria, with new task types such as security patches and performance work. More than 20 open-source maintainers built the tasks from their own repositories, each spending over 40 hours per task. Cognition does not give the total number of tasks.

The full leaderboard

The blog lists eight models, each at its best reasoning effort. Every one of them scores lower on 2.0 than on 1.1.

FrontierCode 2.0 leaderboard vs 1.1: Claude Opus 5.5 45.5% (54.6%), GPT-6 Astra 42.8% (53.3%), Grok 4.7 28.6% (47.6%), Gemini 4 Argon 26.3% (47.5%), Muse Wonder 1.0 24.3% (47.0%), SWE-2 23.9% (50.0%), DeepSeek V4.1 Flash 19.3% (43.9%), Kimi K3 17.7% (44.2%); Cognition, vendor-reported

Opus 5.5 leads at 45.5%, ahead of GPT-6 Astra at 42.8%. The gap to third place is wide: Grok 4.7 scores 28.6%, and Gemini 4 Argon 26.3%. On 1.1 these models sat within about seven points of each other, so 2.0 spreads them much further apart.

The drop of about nine points for Opus 5.5 fits Cognition's claim that the benchmark is far from saturated. Others fell harder. Grok 4.7 lost 19 points, and Cognition's own SWE-2 fell from 50.0% to 23.9%. The open models at the bottom, DeepSeek V4.1 Flash and Kimi K3, score 19.3% and 17.7%. The blog text rounds the top score to 45%.

Where models fail

The blog also breaks failures down by type of blocking criterion. The figures are the mean of Opus 5.5, Fable 5.1, GPT-6.1 Sol and GPT-6 Astra at high reasoning effort.

FrontierCode blocker failure rates, 1.1 to 2.0: thoroughness 11.7% to 19.2%, core behavior 9.5% to 14.9%, backwards compatibility 2.9% to 5.9%, avoiding regressions 1.3% to 3.3%; scope 9.0% to 3.8%, test integrity 3.4% to 2.5%, effective test writing 2.8% to 1.6%, lint 1.7% to 1.0%; Cognition

Thoroughness is now the most common reason a solution scores zero, at 19.2% against 11.7% on 1.1. Core behavior failures rose from 9.5% to 14.9%. Scope failures moved the other way after the rule change. In short, 2.0 punishes unfinished work more than it punishes doing something unexpected.

Cognition gives two sample tasks to show this. In a Mock Service Worker task, Opus 5.5 failed all five runs at every effort level because its fix skipped the user's cleanup code. In a PostHog Android SDK task, GPT-6 Astra passed five of five runs at low effort but only one of five at high effort, where it deleted a public API class. During the audits an agent also found a security flaw in one upstream project, which was reported to its maintainers.

How others read it

AlphaSignal's write-up adds a cost angle. It puts GPT-6.1 Sol at 39% for $1.03 per task, the best result for the money in the report. It also notes that Muse Wonder 1.0 and Gemini 4 Argon, which do well on Cognition's DeepSWE v1.1, land mid-field here.

The same outlet points out that Cognition builds both the benchmark and the leaderboard, so teams should run their own trials. That concern is not new. When FrontierCode 1.0 launched in June, Latent Space reported that Theo questioned its variance and reproducibility, and Cognition responded.

The benchmark is gaining outside use. Artificial Analysis made FrontierCode v1.1 part of its Coding Agent Index v2.0 on Oct 23, replacing DeepSWE. Mercor hosts a FrontierCode leaderboard in partnership with Cognition, built on 150 public tasks.

What to keep in mind

FrontierCode v1.1 uses a private test set of 100 software engineering tasks on real open-source repositories, according to Artificial Analysis. Tasks are graded on whether a maintainer would merge the change. Artificial Analysis reported a different Opus 5.5 score on 1.1 than Cognition did, 56.0% against 54.6%, which points to harness differences.

A score on this benchmark is only comparable when the harness and rubric match. The published criteria and sample tasks are the way to check that.

So does FrontierCode 2.0 show that AI coding agents are not ready for real projects? It shows they are not ready to work unattended. The best model gets less than half the available score, and thoroughness failures alone zero out almost one task in five. Until outside groups run the same rubric with their own harness, the exact ranking stays Cognition's word, but the drop from 1.1 is large enough to take seriously.