PC Slower Than It Used to Be?
A free scan shows the junk files, broken settings and background clutter dragging Windows down - then fixes them in one click.Free scan · Windows 10 & 11Outdated Drivers Are Slowing You Down
One free scan finds every outdated or missing driver and matches the right update for your exact hardware.Free scan · exact hardware matchiTechGuides is reader-supported. When you buy through links on our site, we may earn an affiliate commission. As an Amazon Associate I earn from qualifying purchases. Learn more
Two mutation-testing reports can both show a score of 1.000 and still measure different things. The score means every mutant counted under that report’s rules was defeated; it does not mean the reports included the same mutants. Compare each formula, denominator population, mutant statuses, and run scope before treating the scores as equivalent.
What a mutation score of 1.000 tells you
A mutation-testing tool makes small changes to a program, called mutants, then checks whether the test suite detects those changes. A mutant is typically considered “killed” when a test exposes behavior that differs from the original program.
In the conventional definition, the mutation score is the number of killed mutants divided by the number of non-equivalent mutants. A score of 1.000 means every mutant in that denominator was killed. It is a statement about the counted set—not proof that the test suite detects every possible defect in the software.
The denominator matters because it defines which mutants the score evaluates. One report may include a broader population than another, even when both display the same result.
Why the denominators can disagree
All non-equivalent mutants versus covered mutants
The conventional score uses non-equivalent mutants as its denominator, including mutants that tests did not cover. A covered-code mutation score uses a narrower denominator: mutants covered by tests. The latter asks whether the tests defeated the mutants they reached, rather than whether they defeated the full non-equivalent population. These are distinct measures, even if both happen to equal 1.000. The conventional definition and the covered-code formula describe these different denominator choices.
Equivalent mutants and exclusions
An equivalent mutant behaves like the original program for the relevant inputs, so tests cannot distinguish it from the original. Conventional formulas exclude equivalent mutants. But identifying equivalence is difficult, and the general problem cannot be solved automatically; tools may classify, exclude, or leave unresolved different mutants. As a result, two reports may not agree on which mutants belong in the denominator. The thesis discusses this complication, and the Luxembourg mutation-testing repository describes the limits of automatic equivalence detection.
Status rules for timeouts and errors
Reports may also differ in how they treat outcomes beyond a clear kill or survival. In the thesis’s covered-code formula, “defeated” mutants include killed, timed-out, and error outcomes. That rule should not be assumed for every tool or report. Check whether timeouts, errors, uncovered mutants, and skipped mutants count as defeated, excluded, or something else. The covered-code definition specifies its status rule.
Different run scopes or mutant selection
Even reports from the same tool can evaluate different populations if one run targets a project and another targets a module, changed code, or selected operators. Sampling, filtering, or incremental execution can also change which mutants are present. The score alone does not reveal whether the two runs used the same scope or selection.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.How to reconcile the two reports
For each report, record the items below. Use the tool’s report output or documentation where available; the title alone does not identify the tools, versions, projects, or counts involved.
- Formula: Is the score conventional, covered-code, or tool-specific? Copy the exact numerator and denominator definitions.
- Run scope: Note the project, module, changed code, or operator selection covered by the run.
- Mutant counts: Capture how many were generated, killed, survived, uncovered, excluded, or judged equivalent.
- Status handling: Record how the report classifies timeouts, errors, skips, and other nonstandard outcomes.
- Selection and execution: Check whether sampling, filters, or incremental execution altered the mutant population.
Then compare the counts and formulas—not just the displayed decimals. If the denominators include different populations, report the results as different measures, even when both display 1.000. Without the specific reports and their tool/version documentation, it is not possible to determine which denominator convention either one used.
Quick Recap
Best Value
Rank #4
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
Recommended Free Tools
