P(doom) represents the subjective probability that artificial intelligence will cause existentially catastrophic outcomes for humanity. The term has evolved into a serious metric used by researchers, policymakers, and industry leaders to express their assessment of AI existential risk. The exact scenarios encompassed by "doom" vary but generally include human extinction, permanent disempowerment of humanity, or civilizational collapse (Field, 2025).
Quantifying existential risk faces fundamental challenges due to the unprecedented nature of the threat. Unlike other risk assessments that can draw on historical data or empirical evidence, AI existential risk estimates rely heavily on theoretical arguments, expert judgment, and reasoning about future scenarios that have never occurred. There is no standardized methodology for calculating P(doom) - each estimate reflects the individual's subjective assessment of factors like AI development timelines, alignment difficulty, governance capabilities, and potential failure modes.
Expert estimates vary dramatically, spanning nearly the entire probability range. A 2023 survey found AI researchers estimate a mean 14.4% extinction risk within 100 years, but individual estimates range from effectively zero to near certainty (PauseAI, 2025; Field, 2025):
- Roman Yampolskiy: 99.9%
- Eliezer Yudkowsky: >95%
- Dan Hendrycks: >80%
- Paul Christiano: 46%
- Holden Karnofsky: 50%
- Yoshua Bengio: 20%
- Geoffrey Hinton: 10-20%
- Dario Amodei: 10-25%
- Elon Musk: 10-30%
- Vitalik Buterin: 10
- Yann LeCun: <0.01%
- Marc Andreessen: 0%
The wide variation in estimates highlights several important limitations. First, many experts don't specify timeframes, making comparisons difficult. Second, the definition of "doom" varies between existential catastrophe, human extinction, or permanent disempowerment. Third, estimates are highly sensitive to assumptions about AI development trajectories, alignment difficulty, and institutional responses. While we cannot access any "objective" probability of AI doom, even subjective expert estimates serve as important inputs for prioritization and policy decisions. The substantial probability mass that knowledgeable experts place on catastrophic risks—including those who developed the AI systems creating these risks—suggests the risk scenarios described in this chapter deserve serious attention rather than dismissal as science fiction.