Skip to content

Latent space, not language: how agents cut inference cost 75%

Two mech figures shake hands through a glowing purple portal above a broken megaphone with scattered letters AI agents

Researchers built a multi-agent framework that cuts inference cost by 75% – without language between agents.

Our AI host Mira found a framework built by researchers from Stanford, MIT & Nvidia. In it, multi-agent AI systems skip language entirely: agents pass compressed meaning directly through latent space instead of words.

Results:
• +8.3% accuracy
• 2.4× faster inference
• 75% fewer tokens
• $4.27 training cost.

If you're routing natural language between every agent node, the communication layer is your bottleneck

0:00
/0:53
thehype.radio deep dive on RecursiveMAS, based on a post by Paul Iusztin

What RecursiveMAS is

The framework is called RecursiveMAS. It comes from a team at UIUC, Stanford, Nvidia and MIT, led by Jiaru Zou and James Zou. The paper went up on arXiv on April 28.

In a normal multi-agent system, each agent writes text and the next agent reads it. RecursiveMAS skips that step. Agents pass their hidden states to each other through a small module called RecursiveLink.

The base models stay frozen, and only the RecursiveLink modules are trained. That keeps training cheap: $4.27 in the paper's setup, compared with $6.64 for LoRA and $9.67 for full fine-tuning.

What the numbers mean

The team tested RecursiveMAS on 9 benchmarks in math, science, medicine, search and code. The agents were small open models with 1B to 9B parameters, such as Qwen3.5, Llama 3.2 and Gemma 3.

The speed and token numbers in the video are the best results, not the averages:

  • Accuracy – +8.3% on average over single-agent, multi-agent and recursive baselines
  • Speed – 1.2x to 2.4x faster end to end
  • Tokens – 34.6% to 75.6% fewer

The paper tests four ways to organize agents. A planner, a critic and a solver work in sequence. Domain experts pass results to a summarizer. An expert teaches a learner. A reflector works with a tool caller.

The trade-off is visibility. Text messages between agents can be read and logged. Hidden states can't, so a failed run is harder to debug.

Update, October 16, 2026: full code, data and NeurIPS

In June, the repo had model checkpoints and demo inference. On June 28, the team released the full training and evaluation code and the training data.

The RecursiveMAS repo now has 968 stars and 117 forks under the MIT license. The authors list the work as a NeurIPS 2026 paper. A second version of the paper came out on July 13 with the same headline numbers.

Sources

Portrait of Nick Trenkler, author at thehype.news

Nick Trenkler

Contributing Journalist & Content Creator at thehype. Nick holds a Bachelor’s degree in Journalism, Master’s in Marketing. He has been leading marketing for AI startups since 2024, with experience in media, digital strategy, and tech-focused content.

More articles by Nick Trenkler →
On air · radio.thehype.news ↗ AI news radio — 24/7