Skip to content

Google launches Gemma 4 12B – nearly matches the 26B model on benchmarks, sometimes beats it, at less than half the memory footprint

Surgeon cracking open a chip labeled LLM Backbone with Vision and Audio modules wired in, monitor showing 12B Models


What changed under the hood:

• Vision. Replaced the encoder with a lightweight embedding module (single matrix multiply + positional embedding + normalization). The LLM backbone now handles visual processing directly

• Audio. Encoder removed entirely. Raw audio signal is projected straight into the same token space as text

• Inference. Ships with multi-token prediction (MTP) drafters for speculative decoding, cutting latency

Benchmarks (Gemma 3 27B / Gemma 4 12B / Gemma 4 26B):

- GPQA Diamond: 44 / 78.8 / ~80
- BBEH: 18 / 53 / 62 MMLU Pro: 67 / 77.2 / 78
- LiveCodeBench: 28 / 72 / 76
- DocVQA: 83 / 94.9 / 93
- InfoVQA: 60 / 88.4 / 90
- MMMU Pro: 65 / 69.1 / 72

Runs locally on consumer laptops with 16GB VRAM or unified memory – including MacBook M-series

0:00
/0:41

Demo source: Google

Portrait of Addy Crezee, author at thehype.news

Addy Crezee

Founder, CEO & Editor-in-Chief at thehype. Addy is a serial entrepreneur building products and communities since 2014. In 2024, he founded /function1, one of the largest AI conferences in the world, scaling from 0 to 10k attendees in its first year.

More articles by Addy Crezee →
On air · radio.thehype.news ↗ AI news radio — 24/7