Bayesian inference updates what is uncertain about a hypothesis or quantity when new data arrive. It combines a prior distribution with a likelihood to produce a posterior distribution: the updated account of uncertainty given the data and the model.
What Bayesian inference means
Bayesian inference is a method for reasoning about an unknown quantity or hypothesis using probability distributions. A distribution represents uncertainty across the possible values or hypotheses. After observing data, Bayes’ rule updates that distribution rather than returning only a single answer.
The result is conditional: it depends on the prior information, the model connecting possible values to the data, and the data that were observed. In other words, Bayesian inference does not remove assumptions; it makes key assumptions part of the analysis.
How Bayes’ rule updates a probability
In words, Bayes’ rule says:
posterior = (likelihood × prior) / evidence
In proportional form, the same relationship is written as:
Free tools Windows power users keep installed
One-click scans. No signup required.
#1 Best Overall
posterior ∝ likelihood × prior
The product of the likelihood and prior gives each possible parameter value or hypothesis a provisional weight. The evidence, also called the normalizing constant, rescales those weights so the posterior is a valid probability distribution. For a discrete set of hypotheses, it sums the prior-weighted likelihoods across the alternatives; for a continuous parameter, it integrates them.
For a particular hypothesis, the evidence in this equation is the probability of the observed data under the model, averaged over the prior possibilities. It is not simply another name for the likelihood: the likelihood evaluates the data conditional on a specified hypothesis or parameter value, while the evidence accounts for all the possibilities being considered.
Rank #2
What prior, likelihood, posterior and evidence mean
| Term | Meaning | Question it answers |
|---|---|---|
| Prior | A probability distribution over possible parameter values or hypotheses before incorporating the current data. | What was plausible before these data? |
| Likelihood | A model for the probability of the observed data conditional on each possible parameter value or hypothesis. | If this possibility were true, how compatible would the data be? |
| Evidence | The normalizing quantity found by summing or integrating prior-weighted likelihood across the possibilities in the model. | How probable are these data across the possibilities allowed by the model and prior? |
| Posterior | The probability distribution after combining the prior and likelihood and normalizing. | What is plausible after taking these data into account? |
A likelihood is not, by itself, the probability that a hypothesis is true given the data. That latter question is addressed by the posterior, which also depends on the prior and on the alternatives represented in the model.
Why base rates matter
Suppose a test is used to detect an uncommon disease. A positive result can occur both in people who have the disease and in people who do not. The chance that a person has the disease after a positive result depends on three things: how common the disease was before testing, how often the test is positive when disease is present, and how often it is positive when disease is absent.
Rank #3
- Book - bayesian statistics the fun way: understanding statistics and probability with star wars, lego, and rubber ducks
- Language: english
- Binding: paperback
When the disease is uncommon, people without it may greatly outnumber people with it. Even a test that detects disease reasonably well can therefore produce a meaningful number of false positives among the much larger healthy group. A positive test result and the probability of having the disease after that result are different quantities.
Bayesian updating makes this distinction explicit: prevalence contributes to the prior, while the test’s behavior contributes to the likelihood. Without specified prevalence and test-performance values, there is no single numerical probability of disease to calculate from a positive result.
Rank #4
How a Bayesian analysis is carried out
- Define the question. Identify the unknown quantity or the competing hypotheses, and specify what data are relevant to the question.
- Choose a prior. State what information it represents and why it is reasonable. If there is no strong prior information, examine whether plausible alternative priors would change the result.
- Specify the likelihood. Describe how the data could have been generated under each possible value or hypothesis. This is the statistical model linking assumptions to observations.
- Calculate or approximate the posterior. Some models allow a closed-form calculation; others require numerical integration or sampling methods.
- Summarize the uncertainty. Report quantities suited to the question, such as posterior probabilities, quantiles, or credible intervals, rather than presenting a summary as if it captured all uncertainty.
- Check the model’s implications. Use posterior predictions to ask whether the model can reproduce important features of the observed data and whether its predictions are plausible.
- Refine when needed. If checks reveal poor fit or implausible predictions, revisit the prior or model and assess how that changes the conclusions.
How a posterior differs from a point estimate
A point estimate is one selected value, such as the most probable value or a posterior mean. A posterior distribution contains more information: it represents the range of values still plausible under the model and how probability is distributed across them.
Reporting only a point estimate can conceal whether the data strongly concentrate around it or leave substantial uncertainty. A posterior probability or interval can communicate more of that uncertainty, provided the method used to summarize the distribution is stated. A credible interval, for example, is interpreted within the Bayesian model as an interval containing a stated proportion of posterior probability; its meaning is not just “the estimate plus or minus an error bar.”
The Tool Desk
Outbyte Driver Updater FREEFix the driver behind crashes, sound loss and screen glitchesFind Drivers →Outbyte PC Repair FREEClear out junk files and repair common Windows errorsFree Scan →Best Value
- Used Book in Good Condition
What Bayesian conclusions can and cannot establish
A posterior is an updated probability distribution conditional on the prior, likelihood, and observed data. It is not a guarantee that the model describes reality correctly. If the model rules out an important possibility or represents the data poorly, a precise-looking posterior can still mislead.
Prior sensitivity deserves particular attention when data are sparse: in that situation, the prior can have a stronger influence on the posterior. Model checking and posterior prediction help assess whether the assumptions produce reasonable implications, but they do not prove a model is true. Responsible interpretation therefore reports the assumptions and treats the posterior as conditional evidence, not as an assumption-free verdict.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
