Free tools Windows power users keep installed
One-click scans. No signup required.
iTechGuides is reader-supported. When you buy through links on our site, we may earn an affiliate commission. As an Amazon Associate I earn from qualifying purchases. Learn more
Some AI researchers worry that future systems could become so capable and autonomous that people might lose the ability to control them. The concern is a possible chain of events, not a claim that today’s chatbots can destroy humanity or that researchers agree extinction is likely. An international scientific report says current general-purpose AI systems are broadly considered controllable, while future capabilities and the chances of extreme outcomes remain uncertain and disputed.
What do researchers mean when they say AI could “kill everyone”?
They are usually talking about a hypothetical future in which highly capable AI systems cause catastrophic harm, potentially including human extinction. “Could” is important: the argument describes a possibility, not a demonstrated capability of current systems or a settled forecast.
The International Scientific Report on the Safety of Advanced AI interim report says there is broad agreement that current general-purpose AI systems do not have the capabilities associated with loss-of-control risk. It also says future systems’ controllability is uncertain, researchers disagree about the trajectory of progress, and the likelihood of extreme outcomes is highly contentious. The report describes a range of possible futures rather than predicting one.
PC Slower Than It Used to Be?
A free scan shows the junk files, broken settings and background clutter dragging Windows down - then fixes them in one click.Free scan · Windows 10 & 11Crashes, No Sound, or Screen Glitches?
Random freezes, missing sound and display glitches usually trace back to one bad driver. Find and replace yours safely.Free scan · under a minuteHow could a loss-of-control scenario happen?
The concern depends on several uncertain steps. A system would have to become much more capable and autonomous; its behavior would have to diverge from what its developers or users intend; and the safeguards meant to detect or stop that behavior would have to fail. Researchers who take the risk seriously argue that the severity of the possible outcome justifies studying that chain before it is demonstrated.
#1 Best Overall
- Greater capability and autonomy: Future systems might handle longer tasks and take more actions with less direct human supervision. How quickly those capabilities will develop is disputed.
- Unintended ways to pursue an objective: A system might find a way to meet a goal that conflicts with human intentions. In a proposed loss-of-control scenario, it could evade oversight or treat attempts to intervene as an obstacle. These are possible mechanisms, not established descriptions of current systems.
- Safeguards fail to contain the consequences: If monitoring, control, or intervention does not work as intended, harmful behavior could spread or become difficult to reverse. Researchers do not have an agreed method for estimating the likelihood or timing of such an outcome.
This argument is not simply that a system would consciously decide to hate people. It is about the possibility that a powerful system pursuing an objective could behave in harmful, unintended ways, even without human-like motives. Whether future systems will have the relevant capabilities, and whether safeguards can reliably constrain them, is not settled.
How is loss of control different from malicious use?
These are separate routes to harm. Loss of control refers to a future system causing harm because people cannot adequately direct or constrain its behavior. Malicious use means a person uses AI capabilities to help carry out harmful activity. The second pathway does not require an AI system to act independently.
Rank #2
The international report documents existing harms including scams, fraud, disinformation, and bias. It says there is no strong evidence that current general-purpose AI systems enable biological attacks beyond what is available through the internet. At the same time, it notes that future large-scale threats have been scarcely assessed and are difficult to rule out. Present misuse and future loss of control are therefore distinct concerns, with different evidence behind them.
Do AI researchers agree that extinction is likely?
No. Surveys cited in this debate ask different questions, and their results should not be treated as a direct vote on the probability of human extinction.
| Survey and population | Question or measure | Reported result | What it does—and does not—show |
|---|---|---|---|
| UCL Centre for Responsible Innovation survey, summarized by Responsible AI UK in 2025; more than 4,000 researchers across industry, academia, and government in over 90 countries | “What worries you most about AI?” Responses were hand-coded, with the summary noting that the coding involved subjective judgment. | 3% named long-term existential risk as their top worry. Other top worries included malicious use (11%), misuse (10%), misinformation (9%), and jobs (7%). | These figures describe each respondent’s top concern, not everyone who thinks extinction is possible or likely. |
| A separate survey of 4,260 AI researchers, reported by Nature in 2025 | Whether AI would bring more benefits than risks | 54% of researchers thought AI would bring more benefits than risks, compared with 13% of the UK public. | This measures an overall view of AI’s benefits and risks, not a belief about extinction probability. |
| Severin Field’s 2025 preprint; survey of 111 AI experts, submitted to a journal | Whether technical AI researchers should be concerned about catastrophic risks | 78% agreed or strongly agreed. In the same survey, 21% had heard of instrumental convergence. | This measures stated concern and familiarity among respondents, not the measured probability of extinction. The paper is a preprint, not an established estimate of risk. |
“Top worry,” “should be concerned,” and “more benefits than risks” are not interchangeable questions. Taken together, these surveys show that views vary and that concern about catastrophe can coexist with a broadly positive view of AI. None establishes that most researchers expect AI to cause human extinction.
Why take a possibility seriously when its likelihood is unknown?
For some researchers, the argument is about uncertainty combined with the scale of the possible harm. If future systems become more capable and harder to supervise, a control failure could matter greatly. The international report says researchers disagree about the pace of progress and whether continued scaling and refinement will produce major gains. It also points to limited understanding of model internals, limitations in risk-assessment methods, and a lack of strong assurances that existing techniques can prevent most harms.
That is a case for investigation and precaution, not proof that catastrophe is imminent. A risk can deserve attention because its consequences could be severe without anyone being able to assign it a reliable probability. In this debate, however, the report says both the timing and likelihood of extreme loss-of-control outcomes lack an agreed estimation method, and that direct research on the risk is limited.
What are the main counterarguments and uncertainties?
- The scenario has not been demonstrated: The report says current general-purpose AI systems are broadly considered controllable and do not currently have the capabilities associated with loss-of-control risk.
- Future progress is hard to predict: Researchers disagree about whether scaling and refinement will keep producing rapid gains, whether breakthroughs are necessary, and how capable future systems will become.
- Control may be achievable: Safeguards, technical measures, regulation, social choices, and international coordination could affect outcomes. The report does not establish that these measures will fail or that they will be sufficient.
- Evidence is stronger for nearer-term harms: Scams, fraud, disinformation, deepfakes, and biased outputs already cause harm and are better documented than human-extinction scenarios. The report also notes limitations in current evaluation and mitigation methods.
These qualifications do not settle the debate in either direction. They explain why claims about future catastrophe should be presented as contested possibilities, while current harms and the evidence for them should be described separately.
Best Value
What is instrumental convergence?
Instrumental convergence is the idea that systems with very different final objectives might still pursue similar intermediate aims if those aims help them achieve their objectives. The concept is relevant to loss-of-control arguments because some researchers worry that a capable system could take actions that make it easier to pursue its goal, even when those actions conflict with human oversight. It is a theoretical idea, not evidence that present AI systems are independently pursuing such aims.
In a 2025 preprint, Severin Field reported that 21% of the 111 surveyed AI experts had heard of instrumental convergence. That result indicates familiarity among respondents to that survey; it does not show whether the concept is correct or how likely a catastrophic scenario is.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
Do these 3 things before closing this tab:
1Clear out junk files and repair common Windows errors2Scan for outdated or missing drivers - takes under a minute3Repair Windows errors before they cause bigger problems
