What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
In the Carnegie Mellon and University of Illinois course notes covered here, “the fundamental theorem of statistics” refers to the Glivenko–Cantelli theorem: as an independent, identically distributed sample grows, its empirical cumulative distribution function gets uniformly close to the population CDF. The name is not universal, so the intended meaning depends on the author and context.
What does the Glivenko–Cantelli theorem say?
Suppose X1, …, Xn are independent, identically distributed real-valued observations with common cumulative distribution function (CDF) F. Their empirical CDF is
Fn(x) = (1/n) ∑i=1n 1{Xi ≤ x}.
For any threshold x, this is the fraction of observed values that are no greater than x. Under the everywhere-continuous-CDF assumption in the University of Illinois statement, the theorem says:
supx |Fn(x) − F(x)| → 0 almost surely as n → ∞.
#1 Best Overall
“Almost surely” means that the convergence holds with probability one. The statement is stronger than saying the estimate improves at a particular threshold: it says the largest gap across all thresholds tends to zero. The University of Illinois notes give the theorem with the continuity condition; Carnegie Mellon’s notes describe the same central idea.
Why is uniform convergence useful?
At any one fixed threshold, the empirical CDF converges to the population CDF by a law-of-large-numbers argument. Glivenko–Cantelli strengthens that pointwise result: one bound controls the discrepancy across the entire real line, rather than requiring a separate claim for each chosen threshold.
Rank #2
- This guide is a perfect overview for the topics covered in introductory statistics courses.
This makes the empirical CDF a practical stand-in for the unknown distribution when estimating quantities that can be expressed as functionals of a distribution. The Illinois notes give the mean and median as examples of quantities for which plug-in estimation is relevant. But if the goal is to infer a parameter that generated the distribution, that parameter must also be identifiable from the distribution; convergence of empirical distributions does not solve non-identifiability.
What the theorem does—and does not—estimate
The empirical CDF estimates a distribution function, not a smooth probability density. A CDF reports accumulated probability up to a threshold; a density describes how probability is distributed locally. Carnegie Mellon’s notes distinguish empirical CDFs from density estimation, which raises separate estimation choices and trade-offs.
PC Slower Than It Used to Be?
A free scan shows the junk files, broken settings and background clutter dragging Windows down - then fixes them in one click.Free scan · Windows 10 & 11Crashes, No Sound, or Screen Glitches?
Random freezes, missing sound and display glitches usually trace back to one bad driver. Find and replace yours safely.Free scan · under a minuteRank #3
The Illinois notes also state the concentration bound P(‖Fn − F‖∞ > ε) ≤ 2e−2nε², attributing it to Dvoretzky et al. (1956). This is a bound on the empirical CDF’s maximum error under the stated setting, not a survey result or a universal sample-size guarantee.
Why the name is disputed
“The fundamental theorem of statistics” is not a single, universally settled name. The Carnegie Mellon lecture notes say Pitman (1979) calls Glivenko–Cantelli the fundamental theorem of statistics. They also state, without developing the precise assumptions or formulation, “The same kind of result also holds for higher-dimensional vectors.”
Rick Wicklin’s 2014 discussion notes that statistics textbooks do not generally single out one theorem as the fundamental theorem. He considers the law of large numbers and the central limit theorem as candidates and favors the central limit theorem. That is an interpretation of what is foundational, not a reason to reject the course notes’ use of the name for Glivenko–Cantelli.
| Result | Question it answers |
|---|---|
| Glivenko–Cantelli theorem | How closely does the empirical CDF recover the population CDF, uniformly over thresholds? |
| Law of large numbers | Do sample averages or frequencies converge to their population values? |
| Central limit theorem | What approximate sampling distribution describes normalized sums or means? |
These results answer different questions; calling one “fundamental” is a matter of context and emphasis. The phrase should therefore be defined whenever it appears in a statistics discussion.
Do these 3 things before closing this tab:
1Scan for outdated or missing drivers - takes under a minute2Clear out junk files and repair common Windows errors3Fix the driver behind crashes, sound loss and screen glitchesBest Value
A similarly named result is a different theorem
The “fundamental theorem of prevision,” discussed in a 1990 article by Frank Lad, James M. Dickey, and Mohammad A. Rahman, belongs to de Finetti-related work. The article describes its finite form as a computable linear-programming problem and discusses extensions. It is a distinct use of similar wording, not another name for Glivenko–Cantelli.
Further reading
For generalizations of Glivenko–Cantelli and the tools used to prove them, the University of Illinois notes recommend Chapter 19 of A. W. van der Vaart’s Asymptotic Statistics.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
