kimi k3 outperformed claude opus 5 by 57x – on driving the exact same agent

context: not a model-vs-model output test. we benchmarked them as drivers. same task, same builder, same harness – the only variable was who sat in the agent's seat

the setup: @NousResearch's hermes agent cli on @aimlapi. one open-ended prompt: build an explorable 3d room around a computer workstation, single self-contained html file, @threejs via importmap, procedural geometry only, no meshes, no image textures, no layout or style specified. "you are the designer. surprise me"

@Kimi_Moonshot's kimi k3 was the builder in both runs. the driver – the agent that writes the harness, streams the request, waits, opens a browser, verifies and reports – was kimi k3 in run one and @AnthropicAI's claude opus 5 in run two. both returned the same verdict: pass, zero fixes, neither edited a line of generated code

every number below covers the same window: first successful request through end of verification, build call excluded. this is the cost of driving, not of generating

# orchestration cost
• kimi k3 – $0.44
• claude opus 5 – $25.30

# orchestration tokens
• kimi k3 – 646,469
• claude opus 5 – 3,714,005

# billed turns
• kimi k3 – 26
• claude opus 5 – 79

# wall clock
• kimi k3 – 32m
• claude opus 5 – 68m

94.3% of opus's bill was input, not output. across 79 calls it emitted 44,432 tokens, less than the 62,562-token scene it was supervising, and paid $23.85 re-sending a conversation that never grew past 81,467 tokens. every token in that transcript was billed about 45 times. prompt caching was off, and it's provable: a flat $6.50/m in + $32.50/m out fits all 79 paid calls to within $0.00001. with caching the run lands near $3.83

opus also verified harder. kimi looked at screenshots and shipped. opus timed the render loop, then flagged three things that looked wrong and tested each: it read back raw pixels in both lighting modes and found luminance going from 32 to 83, so the day/night toggle worked. it read the source on the lamp's light cone and found a deliberate additive fake-volumetric. it checked the orbit clamps and found them in place. every check came back clean, which is the thing to notice – 68 minutes and $25.30 spent establishing that three non-problems were not problems

# defects that survived verification
kimi k3's scene – 1
claude opus 5's scene – 3

both runs ship the same defect: a desk lamp whose arm passes through a monitor. opus's lamp head sits at (2.0, 1.24, 0.4) inside a panel centered at (2.04, 1.24, 0.34). kimi's arm runs (-0.73, 1.09, -2.32) to (-0.5, 1.26, -2.12) and crosses the left panel at its midpoint. neither model checked its lamp against what was already on the desk. opus adds two more: rain scrolling up the window instead of down, one sign flipped, and unexplained bars across the wall posters. all visible in the first screenshot anyone opened, none caught by the expensive pass, because nothing opus measured asks whether two objects occupy the same space or which way a texture scrolls

conclusion: the per-token rate is not the story. opus 5 input runs $6.50/m against kimi k3's $0.40/m, 16x, but opus also generated 5.75x more traffic for the same job. price times volume is the 57x. the thoroughness premium bought proof that three non-problems were fine, while three real defects shipped