google cloud ai research open-sourced harness², a framework that rewrites the harness around an ai agent while the model weights stay frozen. one refinement step lifted glm-5 from 64.2 to 75.7 on workbuddy-bench. the edits behind that jump were written by claude opus 4.8, an anthropic model. we read the paper, the repo and the benchmark tables.
the paper and the code
the paper “harness²: recursive agent harnessing for an open world” went up on arxiv on october 9, 2026. its nine authors come from google cloud ai research and northwestern university, with tomas pfister as the senior author. the code is on github under the apache 2.0 license.
by october 19 the repo had about 9,300 stars and 821 forks. the readme says it is a research project, not an officially supported google product. the package is version 0.1.0.
a harness is everything around the model that turns it into an agent. harness² can edit eight parts of it: the system prompt, guardrails, memory, tool guidance, subagents, skills, scripts and plugins.
the propose–probe–compose loop
harness² targets the open world: tasks nobody saw while designing the harness, with no grader to say if an answer is right. most self-improvement methods need labels or test cases. harness² compares agent behavior across runs instead.
each task goes through three moves:
- propose – an editor model writes two candidate harnesses. one fixes general failures such as planning and tool use, the other adds knowledge for the task’s domain.
- probe – both candidates run the same task, next to the reference run with the base harness.
- compose – a composer keeps only the edits that the contrast between runs supports.
after each task, a reflector saves the adopted harness and the lessons in a library. the library guides later edits, while every new task still starts from the base harness.
the budget is set by k. parallel mode probes k general and k domain candidates at once. sequential mode chains k refinement steps.
results on three work benchmarks
google tested two executor models, gemini 3.5 flash and glm-5, inside two agent harnesses, opencode and codex. the benchmarks cover professional work: workbuddy-bench (office, web and code), jobbench and harvey’s legal benchmark lab. each score is the mean of three runs.
all four setups improved on all three benchmarks after one step. glm-5 gained the most: after one step it scored 75.7 on workbuddy-bench, above the 72.4 of gemini 3.5 flash with its base harness. the smallest gain was glm-5 under codex on jobbench, from 23.6 to 24.8.
a bigger budget adds more. with five parallel branches, gemini 3.5 flash under opencode reached 80.8 on workbuddy-bench and 39.0 on jobbench. that lifts its gain over the base harness from 5.1 to 8.4 points on workbuddy-bench and from 4.5 to 7.5 on jobbench.
the edited harness also tended to work in fewer steps. on workbuddy-bench, 3 of 4 setups used fewer steps per task. gemini 3.5 flash under opencode went from 25.4 to 21.9.
harness² vs meta-harness, reasoningbank and harness scaling
the paper runs three other approaches on the same tasks. reasoningbank adds retrieved memory to a fixed harness. harness scaling revises the harness per instance. meta-harness searches for one harness offline on separate training tasks.
at k=1 the lead over meta-harness is small: 36.0 vs 35.8 on jobbench and 81.9 vs 81.6 on lab. with gemini 3.5 flash under codex, meta-harness leads on lab, 76.5 vs 75.6. the gap grows with the weaker model: glm-5 under opencode scores 75.7 with harness² and 69.7 with meta-harness on workbuddy-bench.
all baseline scores were measured by the harness² authors. the other methods did not report these benchmarks themselves.
claude opus 4.8 as the editor model
in the main setup, claude opus 4.8 running under opencode proposes and composes the edits. gemini 3.5 flash and glm-5 only do the tasks. so a google paper relies on an anthropic model for the core step.
the paper also tests one model in both roles at k=2. all five models it tried, from glm-5 to claude opus 5, improved on every benchmark. claude opus 5 gained 3.3 points on workbuddy-bench, 5.5 on jobbench and 3.0 on lab. glm-5 gained 13.7 on lab.
ablations: library, scoping, composition
the ablations remove one piece at a time, with gemini 3.5 flash under opencode at k=1. without the lessons library, workbuddy-bench falls from 77.5 to 74.4 and jobbench from 36.0 to 34.2. without the split into general and domain editors, the scores drop to 74.6 and 32.3. without the compose step, they drop to 75.9 and 34.1.
limits: compute, signal and sandbox
the code is free under apache 2.0, but the models it calls are not. the default editor, claude opus 4.8, is a paid model, and the readme routes model calls through a google cloud vertex ai project. each refinement round adds those calls: the paper says every round needs extra runs and editor sessions. parallel sessions cut waiting time, not total compute. token spend appears only in a chart, and the authors call the budget range they tested limited.
the signal itself has limits. without a ground-truth answer, edits rest on contrasts between runs and on model judgment. the authors write that these signals depend on run-to-run variation and shared model biases. they guarantee neither correctness nor transfer to other tasks.
the setup needs care. the readme says executors have unrestricted network access and codex runs with its sandbox disabled. public benchmark rubrics can be found online, so users must add network limits before any evaluation. model runs go through vertex ai and need linux, python 3.12, node.js 20 or newer and the google cloud cli.
sources
- harness²: recursive agent harnessing for an open world – the paper, arxiv, october 9, 2026; tables 1–3, appendix f
- google-research/harness2 – code, readme and setup, apache 2.0
- harness² project page – pipeline figures and the workbuddy-bench results table
