A nonsignificant result does not show that the null hypothesis is true. It means the chosen test did not provide sufficient evidence to reject that specified null under its assumptions and decision rule. A study that fails to detect a difference has not thereby shown that the difference is zero.
What a p-value actually tells you
In a conventional null-hypothesis significance test, the p-value describes how unusual the observed data—or data more extreme—would be if the specified null model were true. It is not the probability that the null hypothesis is true. The National Academies puts it plainly: “The p-value does not represent the probability that the null hypothesis is true.” (National Academies of Sciences, Engineering, and Medicine, 2019.)
Researchers compare the p-value with a decision threshold selected for the analysis. A threshold such as 0.05 is common, but it is not a universal rule; more stringent thresholds such as 0.01 or 0.005 are also used. The threshold and the testing procedure determine whether the result is classified as statistically significant, not whether the null has been proved true or false.
Why “fail to reject” is not the same as “accept”
If the result does not meet the prespecified rejection criterion, the test has failed to reject the null. That conclusion is deliberately limited: the data did not provide enough evidence for rejection under this procedure. The test has not established that the null is correct.
The Tool Desk
Outbyte PC Repair FREERepair Windows errors before they cause bigger problemsFix Now →Outbyte Driver Updater FREEFix the driver behind crashes, sound loss and screen glitchesFind Drivers →#1 Best Overall
- This guide is a perfect overview for the topics covered in introductory statistics courses.
A nonsignificant result can arise because an effect is negligible, but it can also arise because the estimate is imprecise. A study may leave a wide range of possible effects compatible with its data, including effects that would matter in practice. In hypothesis-testing terms, failing to reject a false null is a Type II error; the chance of such an error depends in part on factors such as sample size and the chosen error tradeoff. A large p-value alone cannot tell you which explanation applies.
Nor does a nonsignificant result rule out every alternative explanation. Random error, model assumptions that do not fit the data, and other features of the study can affect the result. A conclusion from a test is conditional on the design, data collection, model, and analysis choices—not an independent certification of a scientific claim.
Rank #2
- Statistions, how to lie
- Darrell Huff
- Illustrated by Irving Genis
- New York - London 5 6 7 8 9 0
What to report instead
Do not make a binary label carry the whole interpretation. Report the estimated effect and its uncertainty so readers can judge both the direction and the range of values the data remain compatible with. State the test and decision criterion where relevant, then describe the outcome without claiming more than it establishes.
- Prefer: “The result did not provide sufficient evidence to reject the null hypothesis.”
- More informative: “The estimated difference was X, with a [confidence interval], and the test did not meet the prespecified significance criterion.” Replace X and the bracketed interval with the study’s actual estimate and interval.
- If the estimate is too uncertain to establish either a meaningful difference or practical equivalence, say that the result is inconclusive about whether a difference exists.
A useful short report can say that the observed group difference did not meet conventional levels of statistical significance, but it should be accompanied by the estimate and an uncertainty interval. Avoid “we proved there is no effect,” “the groups are equal,” or “we accepted the null” when the only basis is a nonsignificant conventional test.
Recommended Free Tools
When equivalence testing is the right question
Sometimes the scientific or practical question is not whether an effect differs from exactly zero, but whether it is small enough to treat as negligible for a defined purpose. A conventional test of a zero-effect null does not answer that question by itself. Equivalence testing addresses it by setting a range of effects considered practically negligible and testing whether the evidence supports an effect within that range.
Choose and justify the margin
Before interpreting results as evidence of equivalence, define the equivalence bounds—the largest positive and negative effects that would still count as negligible in the application. Those bounds need substantive or theoretical justification; they are not chosen simply to make a result pass. The study also needs enough precision to assess whether the effect lies inside them.
Rank #4
- Brand new
- box27
Use a procedure designed for equivalence
One common approach is two one-sided tests (TOST). Interval estimation can also make the comparison clear: the relevant interval must be sufficiently narrow and lie entirely within the prespecified equivalence bounds. A confidence interval that merely includes zero is not enough to establish equivalence.
This distinction matters in treatment comparisons. A conventional test with p > 0.05 does not establish that two treatments are equally effective. If the goal is to assess whether treatments are equivalent—or whether one is not unacceptably worse—use an appropriate equivalence or non-inferiority procedure with a justified margin rather than treating a nonsignificant test of equality as proof (American Association for Cancer Research, 2022).
Free tools Windows power users keep installed
One-click scans. No signup required.
Best Value
How the methods differ
| Approach | Question it asks | What the result supports |
|---|---|---|
| Ordinary null-hypothesis significance test | Are the data sufficiently incompatible with the specified null to reject it under the chosen rule? | Reject or fail to reject. Failure to reject is not proof of the null. |
| Equivalence test | Is the effect small enough to fall within a prespecified, practically negligible range? | Evidence of equivalence requires a justified margin and sufficiently precise data. |
| Bayesian comparison | How do the data compare under specified null and alternative models, given prior assumptions? | The conclusion depends on the models and prior information; it is not the same output as a conventional p-value. |
These approaches answer different questions. A Bayesian result, for example, depends in part on prior probabilities and the chosen alternative model, so it should not be described as if it were a conventional p-value or a direct acceptance of the null.
Statistical significance is not practical importance
Statistical significance is a decision about compatibility with a specified null under a procedure. It does not by itself tell you how large an effect is or whether that effect matters. An estimate can be statistically significant yet practically unimportant; a nonsignificant estimate can still be too uncertain to exclude an important effect. Interpret the estimate, its uncertainty, the study design, and the substantive stakes together.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
