Skip to content

cohere context focus cuts tokens by up to 41%, it says

cohere's context focus trims tokens by up to 41%: a woman slices a stack of books down to one scored passage, 0.94 models

Cohere introduced Context Focus on oct 21, an opt-in feature that returns precise passages with relevance scores instead of whole documents. Cohere says it reduces token consumption by up to 41% while maintaining accuracy.

what cohere announced

The announcement post is a follow-on to a larger launch. The table lists what the sources state.

itemwhat the sources say
featureContext Focus, described as brand-new
outputprecise passages with relevance scores
token claimup to 41% lower consumption, per Cohere
modeopt-in, works like a dial
productRerank 5, in Pro and Fast

The feature is part of Rerank 5, not a standalone product.

how context focus works

Cohere's docs say both Rerank 5 versions support Context Focus. It returns the most relevant passages within each document in the same call. Calibrated relevance scores come with them.

Cohere says the feature works like a dial. The user controls how much context and quality to keep. Because it is opt-in, existing setups are unchanged until it is switched on.

what the 41% figure does not say

The 41% is an up-to figure from Cohere's own statement. The sources give no benchmark, baseline or definition of maintaining accuracy.

The sources also give no pricing for the feature. A team with a retrieval pipeline would need to test it on its own documents to see the real saving.

Until Cohere publishes the method, the number is a best case.

sources

Portrait of Felix Laurent, author at thehype.news

Felix Laurent

News Writer at thehype. Felix is a journalist with experience in marketing. He covers artificial intelligence and emerging technologies, focusing on the ideas, innovations, and trends shaping the future of tech.

More articles by Felix Laurent →
ON AIR · RADIO.THEHYPE.NEWS ↗ ai news radio — 24/7