Addy Crezee
z ai's glm 5.2 – open weights, mit license – ranked #3. google's gemini 3.5 flash came in #4. only anthropic's claude fable 5 (#1) and openai's gpt 5.5 (#2) placed higher
how glm 5.2 scored:
• confirmed success: 9.96% (user explicitly marks the task done via approve/disapprove buttons)
• praise vs complaint: 12.69% (unprompted reactions skew positive – "exactly what i needed" vs "this is broken")
• steerability: 5.45% (after a user correction mid-task, the agent fixes the error and the user accepts the fix)
• bash recovery: 3.60% (how fast it recovers after its own bash command errors out)
• tool hallucination: 1.75% (how often it calls a tool that doesn't exist – invented names, malformed syntax)
now google's gemini 3.5 flash on the same signals:
- confirmed success: 1.06%
- praise vs complaint: 1.71%
- steerability: 1.11%
- bash recovery: 2.68%
- tool hallucination: 1.36%
glm leads on every signal that measures whether the work got done, and stays just as reliable on tool calls – so the gap isn't bought with a reliability tradeoff
now the context. glm 5.2 came out of a 127-person team. google runs ~181,000 employees, a $4.5t valuation, and $400b+ in annual revenue – roughly a 1,400x headcount difference.
a 127-person team shipping open weights outranked one of the most resourced labs on earth on a real-world agentic benchmark
efficiency rules. it's startup era
