Skip to content

Thinking Machines' inkling-small fixed more bugs than Inkling – at 3.5x fewer parameters and 11x cheaper

Cyberpunk illustration of woman between two code monitors labeled 975B and 276B, comparing inkling and inkling-small models Models

Thinking Machines' inkling-small fixed more bugs than Inkling – at 3.5x fewer parameters and 11x cheaper

@thinkymachines by @miramurati shipped inkling-small Jul 30: 276B total, 12B active, against Inkling's 975B/41B. Their card already claims small beats its parent on agentic coding – SWEbench Verified 80.2% vs 77.6%

We wanted to see it on code neither model had seen

The setup: @NousResearch's Hermes Agent CLI by @Teknium1, driven through a PTY so the /goal Ralph loop could run – a judge grades every turn and re-prompts until satisfied. 20-turn budget, file/terminal/browser tools, delegation off so neither could hand work to another model

The apps – two single-file HTML pages we wrote, 4 visible defects each:

1. Tiny town: buildings land mirrored, far ones paint over near ones, autosave fires before the mutation, windows glow at noon.

2. Orrery: worlds drawn inside their own rings, speed slider inverted, clicking a planet opens the next one's card, pause stops nothing.

Neither model was told how many bugs existed – only what correct behaviour looks like

- Bugs fixed

#1 inkling-small – 7/8
#2 Inkling – 6/8

- Wall clock
#1 inkling-small – 4m 41s
#2 Inkling – 32m 57s

- Tokens
#1 inkling-small – 48K
#2 Inkling – 722K

- Cost
#1 inkling-small – $0.10
#2 Inkling – $1.08

- Tool calls
#1 inkling-small – 56
#2 Inkling – 115

Observations:

• Inkling's town run is the story. It found the mirrored-placement bug small missed – then fixed it wrong, swapping the axes correctly but also changing Math.floor to Math.round. We replayed all 100 plots clicking dead centre: 75 land on the wrong plot. It reported "re-checked in the running page, togrid – true"

• Both verify, but only what they already suspect. Each spun up a local http.server and drove the page through the browser console. Neither asked "does a building land where I clicked" – the most visible defect in the app. Small never formed the hypothesis; Inkling formed it, broke it, and confirmed its own break

• 98.9% of Inkling's orrery run was input – 626,890 in against 7,023 out, 27m53s, 69 tool calls, to change 7 lines. Small changed the same 4 bugs in 2m03s and 19K tokens. 33x the tokens for the same 4/4

• The judge never caught the missing bug. It bounced small three times, all on formatting, and said nothing about the fourth defect sitting unfixed. A Ralph loop grades the answer against the goal text, not against the software. Every score above is ours, by hand, against a key neither model saw

Conclusion: at a quarter the parameters, inkling-small fixed one more defect than its parent on a fifth of the money and a fifth of the clock!

0:00
/1:20
Portrait of Addy Crezee, author at thehype.news

Addy Crezee

Founder, CEO & Editor-in-Chief at thehype. Addy is a serial entrepreneur building products and communities since 2014. In 2024, he founded /function1, one of the largest AI conferences in the world, scaling from 0 to 10k attendees in its first year.

More articles by Addy Crezee →
On air · radio.thehype.news ↗ AI news radio — 24/7