iTechGuides is reader-supported. When you buy through links on our site, we may earn an affiliate commission. As an Amazon Associate I earn from qualifying purchases. Learn more
A CI check can report success even when its JSON findings are empty because it may be validating only a count—or an exit code—rather than checking that the report contains the data it needs. In the DocsWatcher incident described by its author, a GraalVM native binary emitted empty objects for findings, so the GitHub Action counted no breaking changes and passed.
Why did the check pass when the JSON contained empty objects?
DocsWatcher scans code for API calls with announced shutdown dates, including OpenAI models and Stripe API versions, according to the tool’s author. Its GitHub Action runs the CLI, reads a JSON report, counts findings with breaking severity, and fails when that count is above zero.
In the reported incident, the native executable serialized each finding as an empty object: {}. Because the expected fields were missing, the Action had no finding marked breaking to count. It interpreted missing data as zero breaking findings and returned a green result. A green check therefore meant only that the Action’s condition was not met—not that the report had been validated or that the scan had found no breaking changes.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
What caused the fields to disappear?
The author traced the behavior to the difference between running on the JVM and compiling with GraalVM Native Image. The report says Jackson accessed Java record accessors reflectively, but the Finding record’s accessors had not been registered for reflection in the native image. Without the required reachability metadata, those fields were absent from the serialized output.
#1 Best Overall
The reported remedy was to add reachability metadata listing the record accessors, including severity and change. The author also reports a related failure for findings with multiple locations: the Evidence[] array type needed registration too. That illustrates why checking one ordinary finding may not cover every serialized shape.
Why did the tests miss it?
JVM unit tests exercised a different runtime
The author says the unit tests ran on the JVM, where the reflective access worked. Those tests did not reproduce the native image’s closed-world reachability constraints, so they could pass while the shipped binary omitted the fields.
The native smoke test checked status, not the report
The release smoke test did run the native binary, but it checked only that a repository containing a breaking finding caused exit code 1. The CLI returned the expected code, so the smoke test passed even though its JSON output was empty. Validating a process status and validating its output are separate checks.
The Tool Desk
Outbyte Driver Updater FREEFix the driver behind crashes, sound loss and screen glitchesFind Drivers →Outbyte PC Repair FREEClear out junk files and repair common Windows errorsFree Scan →How should a CI test catch this class of defect?
The incident’s practical lesson is to test the artifact users actually execute and assert both its status and the structure and content of its output. A test that checks only a successful or failing exit code can miss malformed JSON that the next step interprets as clean.
- Run the native artifact: Test the compiled binary on the relevant platform runner, not only the JVM version.
- Parse the JSON: Assert that each finding has the required fields, including severity, rather than merely checking that the output is syntactically valid.
- Check expected content: Use a fixture with a known breaking finding and assert that the report contains
breakingseverity. - Fail closed on malformed findings: Make the Action reject a finding with missing severity instead of treating it as a non-breaking result.
- Cover varied shapes: Include findings with multiple locations so array and nested types are exercised.
The author reports that the fix combined reachability metadata with tests that inspect the JSON from the native executable and reject findings lacking severity. The source result describes that fix as present in v0.3.0 and later, and says the v0 tag points to the fixed release. Those release details are the author’s claims and were not independently verified, so check the project’s current release information before relying on them.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
