Recommended Free Tools
Use a per-test false-positive rate to describe the chance of rejecting a true null hypothesis in one test. Use false discovery rate (FDR) control when testing a family of hypotheses and you want to limit the expected share of false results among the findings you report. They measure different errors, so the right choice depends on the question, the consequences of a false result, and how the tests relate to one another.
What each rate measures
In statistical hypothesis testing, a false positive is a rejection of a null hypothesis that is actually true. For one test, the probability of this Type I error is described by the test’s significance level, often written as alpha. NIST explains significance level as the risk of rejecting the null when it is true: NIST: What are statistical tests?
For a family of tests, let V be the number of false rejections and R the total number of rejections. The false discovery rate is the expected proportion of rejections that are false, conventionally written as E[V/R], with the proportion defined as zero when there are no rejections. Benjamini and Hochberg introduced this criterion for multiple significance testing in their 1995 article, Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing.
| Measure | Question answered | Denominator or event |
|---|---|---|
| Per-test Type I error / false-positive rate | For a true null hypothesis, how often would this test reject it? | One test, conditional on its null being true |
| False discovery rate | Across repeated use of a procedure, what is the expected share of declared findings that are false? | All rejected hypotheses in the defined family |
| Familywise error rate (FWER) | What is the chance of at least one false rejection in the family? | The event that any rejection is false |
FDR is not the probability that a particular reported finding is false, and it is not the probability that any result in the family is false. The latter question is closer to FWER. The Benjamini–Hochberg paper explains that FDR equals FWER when all tested hypotheses are true and is smaller otherwise.
#1 Best Overall
When to use a per-test error rate
One pre-specified test
If your analysis has one planned hypothesis test, report the test and its significance level, and describe that level as the Type I error risk for the procedure. Do not interpret alpha as the probability that a significant finding is false: that would condition on observing a significant result and also depends on how common true alternatives are and on the testing process.
False rejection has high consequences
Choose the test threshold and study design in light of the cost of a false rejection. A significance level describes a risk criterion; it does not by itself establish that a design is appropriate or that the result is practically important.
Rank #2
- This guide is a perfect overview for the topics covered in introductory statistics courses.
When to control FDR
Many hypotheses and a list of findings
FDR control may suit an analysis that tests many candidates and produces a list of discoveries, when the objective is to manage the expected proportion of false findings among those declared. The procedure applies to a defined family of tests, not to an isolated result considered on its own.
Several planned comparisons
For multiple comparisons, define which hypotheses belong to the family before interpreting adjusted results. The National Center for Education Statistics’ 2002 standards list FDR alongside Bonferroni, Scheffé, and Tukey procedures as options to consider for multiple comparisons: NCES Statistical Standard 5-1. The appropriate method depends on the inferential goal and the design.
Rank #3
When FDR is not the right guarantee
If even one false rejection in the family would be unacceptable, FDR is not a guarantee that no false positive will occur. Consider whether a familywise error criterion better matches that requirement. The choice is about which error event matters: an expected false share among discoveries or the chance of any false discovery at all.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Check assumptions and define the test family
A correction only supports the guarantee of the procedure under its stated conditions. The original Benjamini–Hochberg result establishes FDR control for its sequential procedure when test statistics are independent; it should not be assumed to guarantee control under arbitrary dependence. If tests are dependent, identify that structure and use a method whose guarantee covers it.
Rank #4
- Specify the family of hypotheses being tested.
- State the error criterion and target level.
- Name the correction or testing procedure.
- Explain relevant assumptions, including dependence among tests and validity of the tests or p-values.
- Say whether the analysis is exploratory or confirmatory.
“False positive” also has meanings in areas such as security and diagnostic testing. Here, the definitions concern statistical hypothesis tests; NIST’s glossary includes broader, domain-specific uses: NIST False Positive glossary.
Quick Recap
Best Value
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
