Do these 3 things before closing this tab:
1Repair Windows errors before they cause bigger problems2Scan for outdated or missing drivers - takes under a minute3Clear out junk files and repair common Windows errorsiTechGuides is reader-supported. When you buy through links on our site, we may earn an affiliate commission. As an Amazon Associate I earn from qualifying purchases. Learn more
If tests for changed code appear to have been skipped, first determine whether the test executable ran at all. A job that never started points to a CI changed-file condition; a running Catch2 executable points instead to test-case selection, section or generator paths, or runtime skipping. These are different mechanisms, and without the workflow, invocation, output, and Catch2 version, none can be identified as the cause of a particular incident.
First find where the skip happened
Use the job log to establish whether the runner launched the test command and whether the test executable started. The location narrows the investigation:
| What happened | What may be filtering | Where to look |
|---|---|---|
| The test job did not start | A workflow path filter or a job-level condition based on changed-file detection | Workflow YAML, change-detection outputs, and CI logs |
| The executable started, but some tests or sections did not run | Catch2 test-case selection or execution-path filtering | Exact command line, Catch2 version, section/generator structure, and test output |
| The executable ran and reported a skip | Catch2 runtime SKIP(), or a narrower environment-specific issue |
Test output, skip calls, triggering sequence, compiler and standard-library versions |
A “skipped” label alone does not distinguish these cases. In particular, a test never selected by CI is not the same as a section skipped after Catch2 begins executing.
How Catch2 path filtering can leave code unrun
Catch2 represents sections and generators as execution paths through a test case. Its execution-path filtering documentation says path filters match prefixes of paths and are independent of test-case selection: “Path filters are independent of test case selection, Catch2 will try to follow the path filters in all selected test cases.” In practice, a path filter can be applied across selected test cases, and a filter that matches only a higher-level section or generator path may leave descendant paths unfiltered.
That behavior makes nesting depth important: compare the active filter with the actual section and generator tree, rather than assuming that a filter names one complete leaf path. Check the documentation for the project’s Catch2 release before relying on syntax or behavior described on the development-branch page; releases may differ.
Section selection is a separate control
Catch2’s command-line documentation describes -c and --section for selecting sections. Repeating the option can narrow selection to nested sections, for example -c sa -c sb. Selecting a parent section runs its nested sections. Code outside sections still executes, including setup before the first section, so its presence does not prove that every section ran.
When diagnosing this path, capture the entire invocation: test-case filters, every section option, and any execution-path filter. Compare those arguments with the test’s nested sections and generators.
If the test job never started, inspect CI changed-file rules
GitHub Actions workflow-level paths and paths-ignore filters decide whether a workflow runs based on changed files. A job can also be gated by an if condition that consumes outputs from a change-detection action. Inspect both levels: a workflow may start while its test job is conditionally skipped.
GitHub’s workflow syntax documentation explains the path-pattern rules and comparison behavior. Pushes use a two-dot comparison and pull requests use a three-dot comparison. The documentation also describes limitations for pushes of more than 1,000 commits, diff generation that times out, and diffs containing more than 3,000 files; under these conditions, path filtering may not behave as expected. GitHub further warns that a workflow skipped by path filtering can leave its associated check pending, which may block a pull request when that check is required.
If the workflow uses the paths-filter action, inspect its configured base, patterns, exclusions, outputs, and the job or step conditions that consume those outputs. Its README describes different change-detection behavior by event: pull requests are compared with the PR base using the GitHub REST API; feature-branch pushes use a merge base with the configured or default base branch and require a checkout. Do not assume that the action and GitHub’s workflow-level path filters use the same comparison or are configured identically.
Rank #4
Distinguish runtime skipping from path selection
Catch2’s runtime skip documentation covers SKIP() in sections and generators. A section or generated value can be skipped while other execution continues, and the test case may be reported as skipped; a failing assertion takes precedence in the report. This is a runtime outcome, not evidence that an execution-path filter or CI glob excluded the test.
The Tool Desk
Outbyte PC Repair FREEClear out junk files and repair common Windows errorsFree Scan →Outbyte Driver Updater FREEScan for outdated or missing drivers - takes under a minuteDriver Scan →Catch2 also documents a narrower standard-library runtime issue in its known limitations: an exception-handling bug in certain standard-library versions can cause later sections to be skipped after CHECK_THROWS. Treat this as a targeted possibility only when the test sequence and toolchain match the documented conditions. Gather compiler, standard-library, and runtime versions before attributing unexplained section skips to it.
Best Value
A practical diagnostic sequence
- Confirm whether the test executable started. If not, inspect workflow path filters and job-level
ifconditions. If it did, use its output to investigate Catch2 selection or runtime behavior. - Record the exact test invocation and Catch2 version. Include test-case filters, repeated
-c/--sectionoptions, and execution-path filter syntax; compare the filter depth with nested sections and generators. - For CI filtering, inspect the actual changed-file list. Check the exact glob and exclusion rules, comparison base, triggering event, and any action output used in conditions.
- Read the test report precisely. Separate a runtime
SKIP()report from a test that was never selected or a job that did not run, and note whether failures affect the final status. - Check the toolchain only when the evidence fits. Compare the compiler, standard library, runtime, and sequence around
CHECK_THROWSwith Catch2’s documented limitation.
A concrete root-cause finding requires the relevant workflow or CI condition, changed-file list, test command and output, and—if Catch2 is involved—the test structure and version. Without those artifacts, the evidence supports these diagnostic branches, not a claim about which filter skipped a specific test.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
