hermes 2.3x faster but not cheaper: openclaw extraction benchmark

hermes vs openclaw on web extraction

hermes shipped an update claiming web extract got up to 60x faster and 49x cheaper – scraping backends passing cleaner content straight to the agent, big pages handled without the redundant processing. bold claim. so we put it head to head against openclaw

both ran fully local. gemma-4-e4b-it (unsloth iq4_xs gguf) on a macbook air m3, 16gb ram. one task – extract the wikipedia page on claude mythos

openclaw
- 20k tokens
- 2:59
- full structured extraction in one shot: overview, history, specifications, vulnerabilities, responses, the whole mythos timeline laid out

hermes
- 22.9k tokens
- 1:17
- browser_navigate + snapshot, compressed to a clean overview, then stopped to ask if you want the full text

thoughts:
• the speed claim held. hermes was ~2.3x faster
• the cheaper claim didn't really show here – it actually spent slightly more tokens (22.9k vs 20k), because it front-loads a summarize step instead of dumping the page raw. you pay a little up front to keep the main loop lean. different optimization, not a free lunch