Do these 3 things before closing this tab:
1Repair Windows errors before they cause bigger problems2Scan for outdated or missing drivers - takes under a minute3Clear out junk files and repair common Windows errorsiTechGuides is reader-supported. When you buy through links on our site, we may earn an affiliate commission. As an Amazon Associate I earn from qualifying purchases. Learn more
ZéroJour tests whether a security-advisory agent can answer compound, version-sensitive questions more reliably when advisory data is structured. Its author reports that structured retrieval answered 9 of 11 evaluation questions, compared with 1 of 11 for flat keyword search and 0 of 11 for a model without data tools. Those are project-reported results, not an independently reproduced benchmark—and the evaluation also surfaced reasoning errors that structured data did not prevent.
What ZéroJour is designed to answer
ZéroJour is an experiment in retrieving and reasoning over security advisories, not a general-purpose vulnerability scanner. Its intended questions combine several filters that are awkward to answer with a keyword search alone. For example: which advisories affect exactly axios 1.2.0? Or which advisories require no privileges or user interaction, arrive over the network, score at least 8.0, and what fixes are available for each?
According to author Sam LABBE, the project brings together a corpus of 82 advisories, an agent built with Sanity Context and a local deterministic tool, and an evaluation harness comparing structured retrieval with two controls. The article reports that the corpus covers 16 npm and pip packages, 50 CWEs, and 16 remediation playbooks. LABBE says its data comes from the GitHub Advisory Database and the CISA KEV catalog, excludes withdrawn and malware advisories, and can be rebuilt from live sources. These figures and descriptions reflect the author’s account; they are not independently verified here. Project overview
The Tool Desk
Outbyte Driver Updater FREEScan for outdated or missing drivers - takes under a minuteDriver Scan →Outbyte PC Repair FREERepair Windows errors before they cause bigger problemsFix Now →How the advisory data and agent fit together
Structured advisory records
The described schema stores affected version ranges, CVSS components parsed from the official vector, fix status, exploit maturity, and CWE references. LABBE says exploit maturity is included only when sourced from CISA KEV, rather than inferred. Remediation guidance is kept separately in playbook documents.
#1 Best Overall
Retrieval plus deterministic version checks
The agent connects to a GROQ dataset endpoint and a Knowledge Base endpoint. The design matters because advisory questions require more than finding documents that mention a package: they may require intersecting a version range with a specific release, filtering on individual CVSS properties, checking fix status, or aggregating results by package.
Version comparison is a particularly important implementation choice. GROQ compares version strings lexicographically, which can put “1.10.0” before “1.9.0” when treated as text. ZéroJour’s described implementation therefore delegates version-range checks to a local deterministic semver tool instead of relying on a GROQ query to compare versions. Version-comparison detail
What the reported evaluation found
The project article describes 11 questions run against three approaches using the same question set. The author reports the following scores, which held across two independent full runs:
| Approach | What it could access | Reported correct answers |
|---|---|---|
| Structured Sanity Context retrieval | Structured GROQ and Knowledge Base retrieval | 9 of 11 |
| Flat keyword search | Keyword search over the same documents | 1 of 11 |
| No-tools model control | No data access | 0 of 11 |
According to the article, ground truth was generated independently from seed data using plain predicates and semver, rather than through Sanity. The contrast suggests that structured fields and targeted retrieval can help on this particular set of compound questions. It does not establish that structured retrieval will outperform keyword search on other datasets, question sets, or agents, nor does the small, author-reported evaluation amount to independent validation. Evaluation results
Rank #3
Why structured data did not prevent every error
The two reported repeatable misses are useful because they show where data structure stops helping: the model can still make mistakes when it has to aggregate or order results.
Aggregation across packages
Asked which package had the most advisories scoring at least 7.0, the model selected paramiko. LABBE says the dataset’s answer was pillow, with four qualifying advisories compared with two for paramiko.
Choosing the most recent advisory
For a question about the newest advisory, the model selected a 2018 item because it followed the default storage order instead of sorting by publication date in descending order. The author says prompt changes did not correct either miss. Together, these examples distinguish successful retrieval of relevant fields from reliable multi-step reasoning over the retrieved records. Reported model-side misses
How to replay or refresh the project
The author gives these commands for local use:
npm install && npm run evalruns the evaluation replay.npm run fetch:advisoriesrefreshes the advisory corpus.
These are project instructions reported by the author. They were not run or independently checked here, and corpus contents or commands may change over time. Project commands
Best Value
What the audit journal does—and does not establish
The project article says each query, MCP call, answer, and evaluation verdict is sealed in a local NoireBox journal using a SHA-256 hash chain, Ed25519 signatures, and RFC 3161 timestamps. This is the project’s stated audit mechanism: it describes how the project records activity and timestamps it. It should not be read as a security certification or as independent validation that the journal is secure. Journal description
What readers can reasonably conclude
ZéroJour presents a concrete case for modeling advisory properties explicitly when questions depend on version ranges, CVSS components, fix status, or cross-record comparisons. Its reported results favor structured retrieval over the two controls on the author’s 11-question set. But its own failure examples matter just as much: a structured corpus can make relevant facts accessible without guaranteeing correct aggregation or recency ranking. Treat the scores as a project-specific demonstration, not proof that an advisory agent is dependable for operational security decisions. The author’s summary is: “A keyword search cannot do that. We didn’t argue it, we measured it.” Author’s summary
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
