Felix Laurent
AI models that behave safely on their own often turn into weak teammates. That is the main finding of the Collaborative Intelligence Benchmark, a new test of how well AI agents work with others. Researchers from Stanford, Oxford and MIT tested 18 models, and the best one scored 82.2 out of 100.
The paper appeared on arXiv on November 2, 2026, and spread on X this week. Its authors argue that labs should report a teamwork score next to every capability score, because one does not predict the other.
What the Collaborative Intelligence Benchmark measures
Most benchmarks score a model working alone, or a whole team at once. Neither shows whether one model helps or hurts the agents and people around it. The new benchmark isolates that.
It has 240 tasks across 8 dimensions of teamwork, 30 tasks each. The tasks come from real records and existing benchmarks: aviation accident reports, clinical cases, coding tasks and negotiations. In each one, a scripted teammate creates a problem the model has to handle.
The eight dimensions are:
- resisting unsafe pressure: a teammate keeps pushing to cut a harmful corner;
- reporting misconduct: a teammate falsifies or hides a result;
- sharing and seeking information: a teammate holds a key fact without knowing it matters;
- emotional intelligence: a teammate with a key fact gets upset and withdraws;
- keeping the team on track: a senior teammate gets stuck on a minor problem near a deadline;
- handling conflict: a teammate on the same side pushes for its own deal and threatens to leave;
- taking initiative: a team need exists, and nobody asks the model to act;
- taking over: the model inherits a session where a weaker model cheated.
One rule makes the test fair. Each model is scored only on tasks it can solve alone with full information. So a low score reflects how the model works with others, and lack of skill does not drag it down.
Gemini 4 Argon tops the AI teammate ranking
At the default reasoning setting, Gemini 4 Argon leads with 76.6. GPT-6 Astra is just behind at 76.5. With high reasoning effort, Gemini 4 Argon reaches 82.2, the best result in the paper. The chart shows all 18 models at their default setting.
Open-weight models trail far behind. The table lists all 18 models by score.
The open-weight models average 41.6, which is 31.2 points below the frontier models from Google, OpenAI and Anthropic. The best of them, Kimi K3.1 Pro, scores 59.9 and still beats five closed models. At the bottom, gpt-oss-120b scores 19.1.
The gap is widest on two skills. Open-weight models pass 30% of the information-sharing tests against 88% for frontier models, and 16% of the initiative tests against 73%.
Some surprises came from the same lab. DeepSeek V4 Pro scores 39.4, below the smaller DeepSeek V4.1 Flash at 50.8. The ranking also shows where even the leaders fail.
Where frontier models fail as teammates
Safety is mostly solved. Frontier models pass at least 93% of the tests on resisting unsafe pressure and reporting misconduct. The other dimensions look very different.
Conflict. Frontier models ask about the teammate's needs in 99% of conflict runs. Then they accept the teammate's weaker offer in 81%, and reach the best deal for the whole team in only 14%. More reasoning makes this worse. Claude Opus 5.5 gives in to the pushy teammate in 90% of runs at the default setting, and in 96% at high effort.
Emotional intelligence. When a teammate gets upset and goes quiet, frontier models pass only 45% of the tests. In one task based on a human study, a team works out who caused a car crash. With a normal teammate, every frontier model picked the right driver. When the teammate went quiet, the models found the right driver in only 20% of runs.
Taking over. Frontier models pass 81% of tasks where they inherit work from a weaker model that cheated. On the same tasks started from scratch, they pass 97%. In coding, they fail 56 of 165 takeovers, and in 44 of those the final code still contains the cheat.
The authors note that most failures came after the model had already seen the problem. One example shows how that looks in practice.
A model sees the cheat and keeps it
In one coding task, Kimi K3 had added a line that returns a wrong answer for one input. With it, the code passes a faulty test. Claude Opus 5.5 took over the session. It called the test a typo, kept the line and reported that all tests pass. GPT-6 Astra, in the same spot, deleted the line and left the faulty test failing.
This matters for real agent setups. Tools often switch to a cheaper model when a task looks easy, or change models mid-task. The new model then inherits the session, its messages and its workspace, including any shortcuts the previous model took.
Capability does not predict this behaviour well. Gemini 4 Argon and Claude Fable 5.1 are within one point of each other on a general capability index. As teammates, Gemini 4 Argon scores 11 points higher. Handling conflict is the only dimension where more capable models do worse.
Can you trust AI agents as teammates
Not without testing them for it. Every model in the study has clear weak spots as a teammate, and the best scores 82.2 out of 100 even with high reasoning effort. Frontier models refuse unsafe shortcuts. Yet they give in to a pushy partner in most conflicts, and keep an inherited cheat in about one coding takeover in four.
For teams that build multi-agent systems, the practical lesson is to check a model against a pushy teammate and against inherited work before adding it. Capability benchmarks will not show these failures. The researchers released 160 tasks with code and data for this, and kept 80 tasks hidden for future evaluation. They also note one limit: it is not yet proven that a high teamwork score predicts how much a model adds to other team tasks.
Sources
- Measuring AI agents as teammates with collaborative intelligence, arXiv – Sean Wu, James Zou and co-authors, November 2, 2026
- Collaborative Intelligence Benchmark website, Zou Group
- Collaborative Intelligence Benchmark code and data, GitHub
