Hermes Agent gets a security plugin that tests the bugs it finds

Nous Research's Hermes Agent can now scan a codebase for security bugs. On November 8, 2026, Nous promoted Hermes Security, a new official plugin. It maps how an attacker could reach the code, checks each finding and can write a test that reproduces it.

The plugin is free and open source, and it runs on whatever model the user picks. Part of its code is adapted from OpenAI's open-source Codex Security tool. Its own published test results fall short of the targets Nous set, and a bug report about leaked credentials arrived the day after the announcement.

Nous Research on X: Hermes Agent's new Security plugin

What the Hermes Security plugin does

Hermes Security reviews a repository the way a security engineer would. It starts with a threat model: what an attacker could reach and which inputs the code trusts.

It then splits the code into chunks of about 4,000 lines. Up to 10 reviewer subagents work on them in parallel, with one more reviewer for code that cuts across the project. Every candidate finding is checked against the code and traced to see whether an attacker can actually reach it. Only then does it get a severity rating.

There are three scan modes:

• Standard reads all production code, up to about 360,000 lines.
• Deep makes three passes over the code by default.
• Diff reviews only uncommitted work or a branch's changes.

The plugin also checks dependencies. It compares lockfiles against known advisories with Google's osv-scanner, and flags Node, Python, Go, PHP and Ruby versions that are past end of life.

The most unusual part comes after the review.

How test confirmation works

With test confirmation on, Hermes writes a small reproduction for each finding and runs it in a temporary copy of the repository. A finding is marked "Confirmed by test" only if the reproduction shows the bug and a matching negative test behaves as expected. Tests that never ran are labeled as not run.

Testing a running app needs a separate permission for each scan. It covers one address, one hour and 200 requests, and the agent cannot grant it to itself.

Each report records the evidence with exact quotes from the code. It also lists the findings Hermes ruled out and why, plus any files it skipped. Findings are mapped to the OWASP Top 10 and the ASVS standard. Reports export to Markdown, JSON, CSV and SARIF, the format that GitHub code scanning and most security tools read.

Suggested fixes are read-only, so Hermes does not change the code on its own. How well all this works is a separate question.

How accurate Hermes Security is

Nous published an evaluation file with the plugin. It is small: 26 synthetic test cases, made of 13 pairs of a vulnerable and a fixed version of the same code. Nous itself says the set is "not a genuinely blind holdout."

In the main development run, 81.8% of the bugs Hermes flagged were real, and it found 69.2% of the planted bugs. Across five runs, precision ranged from 0.818 to 0.917 and recall from 0.692 to 0.846. Every run missed Nous's own targets of 0.95 precision and 0.85 recall.

There is good news in the same file. Hermes did not flag any of the fixed, clean versions. Its severity labels never exactly matched the expected ones, though.

Nous planned a head-to-head comparison with OpenAI's Codex Security but has not run it. No real-world findings or CVEs from the plugin have been published yet.

The OpenAI code inside and a credential bug

The plugin's license notice says that parts of it are modified from Apache-2.0 code copyrighted by OpenAI in 2025. Its provenance file points to OpenAI's open-source Codex Security repository. That explains why the pipeline looks so much like OpenAI's: threat model, review, reachability check and reproduction test.

The two differ in where tests run. OpenAI says Codex Security validates findings in sandboxes. Hermes's documentation says its test runs are "not a sandbox": the repository's code runs as the user, with credential-like environment variables removed.

That protection has a gap. On November 9, a GitHub user opened issue #359, "Test harness keeps ambient credentials." The harness filters environment variables by name only. Test code from the repository can still reach the home folder, including AWS and Kubernetes credentials, and the SSH agent. "Allowing tests is consent to execute the code, not to hand over ambient credentials," the reporter wrote. The reporter found the bug by running Hermes Security on its own code.

The issue was still open when we checked. Until it is fixed, test confirmation is safest on repositories you trust or inside a container.

Who built it

The plugin was written by Mark S., who posts on X as @mark_s and describes himself as building Hermes Agent at Nous Research. His bio lists him as a CISSP, a common security certification. He released the plugin on November 5 and asked for feedback. Version 0.1.2 followed on November 9, after about 13 fixes.

Hermes Agent itself is one of the most popular open-source agents. It launched in February 2026 under the MIT license and has more than 260,000 stars on GitHub. It works with models from Nous Portal, OpenRouter, OpenAI and custom endpoints. Hermes Desktop, its app for macOS, Windows and Linux, came out in June.

Hermes Security joins a crowded field.

How it compares with other AI security tools

Big AI labs have shipped similar tools over the past year:

Hermes Security is the only one of these that is fully open and runs locally inside a general-purpose agent. It also leaves the choice of model to the user. It also keeps rejected findings and coverage gaps in every report. In exchange, it has the smallest published evaluation and no sandbox.

How to try Hermes Security

The plugin needs Hermes Agent 0.21.5 or later and Python 3.11 or later. Install it with:

hermes plugins install hermes-security --enable

You can then ask in chat to scan the repository, use the /security command, or run a scan from the terminal:

hermes security-review start --path my-repo --mode diff --tests

In Hermes Desktop, the Security page is off by default and is turned on under Capabilities, then Plugins. The first scan downloads osv-scanner, about 55 MB. There is no license fee, but every scan uses tokens from your model provider, and your code is sent to that provider.

Is Hermes Security worth using

Yes, as an extra reviewer on code you control. It is free, it explains what it found and what it rejected, and SARIF export fits into existing tools. Diff mode makes it practical before each merge.

It should not replace a dedicated scanner or a human review yet. Nous's own tests show it misses about one bug in four and falls short of its targets. Keep test confirmation off on untrusted code until the credential issue is fixed, and read every finding before acting on it.

Sources

• Nous Research on X – November 8, 2026
• Hermes Security plugin, Hermes Agent marketplace
• hermes-security README and evaluation, GitHub – November 2026
• Test harness keeps ambient credentials, issue #359, GitHub – November 9, 2026
• OpenAI launches Codex Security, The Decoder – March 2026
• Claude Code Security, Anthropic – February 20, 2026