iTechGuides is reader-supported. When you buy through links on our site, we may earn an affiliate commission. As an Amazon Associate I earn from qualifying purchases. Learn more
Current estimates of AI-caused existential catastrophe are too assumption-dependent and weakly grounded in evidence to serve as standalone policy evidence. That does not make probability estimates useless—or justify ignoring catastrophic risks. It means policymakers should treat a number as a conditional judgment, state what it measures, and test whether a decision still makes sense across a range of plausible outcomes.
What does an AI existential-risk probability actually measure?
There is no single, shared event behind every headline estimate. One forecast might ask whether AI causes human extinction; another might count an unrecoverable societal collapse; a third might define catastrophe by a large death toll. Those outcomes are not interchangeable, so their probabilities cannot be compared as if they answered the same question.
![AI Law and Policy: [Connected eBook]](https://m.media-amazon.com/images/I/41Y7jOay2eL._SL160_.jpg)
The Forecasting Research Institute (FRI), for example, posed the question, “Will AI cause an existential catastrophe by 2100?” Its report defines the outcome to include extinction or specified forms of unrecoverable collapse. LEAP Wave 9 instead asked about a global AI-related catastrophe in which more than 10% of the population alive at the start of a five-year period die by its end. That is a severe outcome, but it is not a forecast of human extinction.
A probability also depends on its time horizon and conditions. A forecast conditional on rapid AI progress answers a different question from an unconditional estimate; a 2030 forecast is not a 2100 forecast. Before comparing or using a number, identify the outcome, horizon, scenario, and population whose judgments it summarizes.
What recent forecasts show—and what they do not
The figures below come from different exercises and different questions. They illustrate how estimates vary with the outcome and assumptions; they do not form a like-for-like contest over one event.
| Forecast | Reported estimate | What it measures | Source and limits |
|---|---|---|---|
| FRI adversarial collaboration, 2024 | Skeptical group median: 0.10% at the beginning and 0.12% at the end. Concerned group median: 25% at the beginning and 20% at the end. | AI-caused existential catastrophe by 2100, as defined in the report. | Twenty-two selected participants: 11 “AI skeptic” participants (nine superforecasters and two domain experts) and 11 “AI concerned” domain experts. These are group medians, not a representative survey or established true probabilities. |
| LEAP Wave 9, 2026 | Expert median: 2% under slow progress and 10% under rapid progress. | A global AI-related catastrophe, defined as more than 10% of the population alive at the beginning of a five-year period dying by its end. | Responses were collected May 19–June 10, 2026; the report was released June 30, 2026. The panel included 194 experts, 53 superforecasters, and 612 public respondents. The reported figures are expert medians conditioned on progress scenarios, not extinction forecasts. |
The FRI collaboration found persistent disagreement even after participants spent several weeks reviewing material, forecasting, and summarizing one another’s arguments. Engagement did not bring the two groups’ estimates close together. The report identifies differences over capability timelines, whether AI would develop goals linked to extinction, how difficult human extinction would be, and how societies might respond, alongside broader worldview differences. Short-term indicators examined in the project explained only a modest share of the forecast gap. This is evidence that the disagreement has multiple sources—not that either group is right.
Rank #2
LEAP Wave 9 offers a newer and larger panel snapshot, but it does not settle long-run accuracy. Its difference between slow- and rapid-progress scenarios makes the role of assumptions visible. The numbers record panelists’ judgments about defined future events; they are not observed catastrophe rates.
Free tools Windows power users keep installed
One-click scans. No signup required.
Why the numbers are not validated forecasts of the distant future
For rare events far in the future, there is little direct evidence against which to check a probability. A forecast cannot be called calibrated merely because experts supplied it, because a panel is large, or because the estimate is expressed precisely. The reviewed studies do not establish that century-scale AI catastrophe forecasts are well calibrated, nor do they show that policies selected using those estimates have improved outcomes.
Rank #3
In a May 2026 brief, the Center for Security and Emerging Technology (CSET) argues that evidence and detailed theory are sparse for some catastrophic AI risks. It distinguishes uncertainty caused by ignorance—missing knowledge about mechanisms, scenarios, or likelihoods—from randomness. Andrew Lohn, the brief’s author, writes: “In AI risk, rather than in dice rolls, ignorance is the dominant form of uncertainty, not randomness, so the best techniques are not always probabilistic.”
CSET does not argue that probability is always inappropriate. It discusses belief and plausibility as ways to express how strongly available evidence supports or argues against a scenario. The practical point is that one precise probability can hide how much is unknown. Policy analysis should show the underlying evidence and assumptions, not imply that uncertainty has been measured away.
A separate 2025 preprint by Severin Field surveyed 111 AI experts: 78% of respondents agreed or strongly agreed that technical AI researchers should be concerned about catastrophic risks, and 21% said they had heard of instrumental convergence. Those are findings about that study’s respondents’ views and familiarity with a concept. They do not measure forecast accuracy or establish the probability of catastrophe.
The Tool Desk
Outbyte PC Repair FREEClear out junk files and repair common Windows errorsFree Scan →Outbyte Driver Updater FREEFix the driver behind crashes, sound loss and screen glitchesFind Drivers →How policymakers can use estimates without overclaiming
A probability can still be useful as one input in a decision, provided it is presented as conditional and decision-relevant rather than as a settled fact. A practical policy analysis can proceed in this order:
- Define the harm. Specify whether the policy is intended to reduce extinction risk, prevent unrecoverable collapse, limit catastrophic misuse, preserve human control, or address another outcome. Do not substitute one endpoint for another.
- State the horizon and scenario. Say when the outcome is being forecast and whether the estimate assumes rapid progress, slow progress, or no particular progress path. Label a scenario-conditioned estimate as such.
- Describe the evidence behind the estimate. Distinguish observed data and model outputs from expert judgment, theoretical arguments, and panel surveys. Give the sample and method when they are known, and do not call an opinion survey a calibration test.
- Show uncertainty rather than only a point estimate. Explain where disagreement comes from and which assumptions move the result. Where evidence is especially thin, consider describing what supports or weighs against a scenario instead of presenting a single probability as though it were precise.
- Test the decision across plausible values. Ask what action the estimate would change, what acting or waiting would cost, and whether the policy remains sensible across a broad range of plausible probabilities. This is a decision-analysis implication, not a result tested by the cited forecasting exercises.
- Track observable indicators. Identify developments that would strengthen, weaken, or change the relevant scenario, and specify how they would affect the policy. Indicators should connect to the defined harm rather than serve as vague signals of “AI risk.”
If a policy is sensible across a wide range of plausible probabilities, decision-makers need not resolve a contested point estimate before acting. If the choice changes sharply with the estimate, the analysis should make that sensitivity explicit and explain the assumptions driving it.
What the evidence does and does not settle
The available material supports a limited conclusion: current long-horizon AI existential-catastrophe probabilities are not strong enough to use alone as precise, empirically validated policy inputs. It does not establish that the risk is negligible, that high estimates are reliable, or that policymakers should do nothing. CSET’s methodological critique, FRI’s small structured collaboration, LEAP’s larger scenario-based panel, and Field’s expert survey address different questions; none settles the true probability of AI-caused existential catastrophe.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
