Existential risk

Working Definition

An existential risk (x-risk) is the risk of an event that would either annihilate Earth-originating intelligent life or permanently and drastically curtail its potential for future development. The definition has two prongs, and the second is more consequential than it first appears: it encompasses not only biological extinction, but also an irreversible civilizational collapse or a permanently locked-in totalitarian tyranny from which humanity could never rebuild its future. What separates existential risk from global catastrophic risk (a severe pandemic, a world war, or climate breakdown claiming hundreds of millions of lives) is the criterion of irreversibility: a disaster from which humanity eventually recovers is catastrophic, not existential. In everyday media usage, the term is routinely diluted to mean “anything very dangerous,” stripping it of its analytical force.

Genesis and Global Lineage

Nick Bostrom introduced the term and its conceptual framework in Existential Risks: Analyzing Human Extinction Scenarios and Related Hazards (Journal of Evolution and Technology, 2002; first draft 2001), proposing a fourfold typology (bangs, crunches, shrieks, whimpers) and an initial catalog of threats ranging from molecular nanotechnology and unaligned superintelligence to physics-experiment disasters. The argument reached its mature form in Existential Risk Prevention as Global Priority (Global Policy, 2013), which elaborated the maxipok rule already sketched in 2002: maximize the probability of an “okay outcome,” defined as any outcome that avoids existential catastrophe. Its backdrop included earlier Cold War and fin-de-siècle writing on human extinction—Jonathan Schell (The Fate of the Earth, 1982), John Leslie (The End of the World, 1996), and Martin Rees (Our Final Hour, 2003)—together with the transhumanist-utilitarian school centered on Oxford’s Future of Humanity Institute (2005–2024). Philosophically, the concept builds on Derek Parfit’s argument in Reasons and Persons (1984) that the moral difference between a war that kills 99% of humanity and one that kills 100% is vastly greater than the difference between 0% and 99%. In the 2020s, the concept was captured by two opposing circuits: the AI industry, where “existential risk” became simultaneously a warning and a marketing boast about model power, and critics who group it under the acronym TESCREAL alongside longtermism and transhumanism.

Central European Lineage

John von Neumann articulated a closely related diagnosis half a century earlier. Known to English-language readers primarily as an architect of Princeton mathematics and American Cold War strategy, von Neumann received his formative intellectual training in Budapest, attending the renowned Lutheran Fasori Gimnázium and earning his doctorate in mathematics at the University of Budapest in 1926 alongside chemical engineering studies in Zurich. In his June 1955 Fortune essay Can We Survive Technology?, he framed the problem of human survival at the exact planetary scale later adopted by the existential-risk literature: technological power had caught up with the physical dimensions of the Earth, the destructive reach of offensive weapons invalidated all realistic scales of defense, and “the finite, actual size of the earth” had for the first time become a critical constraint on survival. There is no evidence that this essay directly influenced Bostrom: in his 2002 paper, von Neumann appears only in a footnote via a secondhand remark that nuclear war was certain and would kill everyone. Von Neumann’s Budapest years explain his intellectual formation rather than the immediate context of the 1955 essay, which grew out of American thermonuclear strategy.

Why Now

Between 2018 and 2026, the center of gravity of existential risk shifted from academic philosophy to high-level politics. Toby Ord’s book The Precipice (March 2020) appeared in the very weeks when the COVID-19 pandemic demonstrated how poorly global institutions handle even a moderate catastrophic risk, let alone an existential one; Ord published explicit subjective probability estimates (a 1-in-6 total existential risk this century, including 1-in-10 from unaligned artificial general intelligence, or AGI). The UN Secretary-General incorporated “existential risks” into Our Common Agenda (2021), and the Pact for the Future adopted at the UN Summit of the Future (September 2024) institutionalized the language of intergenerational survival. Following the release of GPT-4 (March 2023), a May 2023 single-sentence statement issued by the Center for AI Safety and signed by the CEOs of OpenAI, Google DeepMind, and Anthropic equated “the risk of extinction from AI” with pandemics and nuclear war, while the intergovernmental Bletchley Declaration (November 2023) warned of serious, even catastrophic harm from frontier models. Meanwhile, the Bulletin of the Atomic Scientists moved its Doomsday Clock to 90 seconds to midnight (January 2023), 89 seconds (January 2025), and 85 seconds (January 2026). Conversely, the collapse of the cryptocurrency exchange FTX (November 2022)—a major funder of x-risk institutions—and Oxford’s closure of the Future of Humanity Institute (April 2024) triggered a fierce backlash over whose political and commercial interests the existential-risk frame actually serves.

Explanatory Power

The concept enforces a distinction that everyday language blurs: between the magnitude of a harm and its irreversibility. First, it reorders policy priorities by the product of probability and irreversibility rather than body count alone, explaining why in Ord’s calculus an asteroid impact (well studied and exceedingly improbable) weighs far less than an engineered pandemic. Second, it explains why markets and electoral politics systematically underinvest in survival: because the benefits of avoiding extinction accrue overwhelmingly to future generations who neither vote nor buy goods today, existential safety is a global public good across time. Third, it exposes an epistemic asymmetry: humanity cannot learn to manage existential risk by trial and error, because the first fatal mistake is also the last, meaning evidence will always remain indirect and contested. That asymmetry explains both the urgency of x-risk researchers and the frustration of critics who demand empirical track records where none can exist by definition.

Axes of Debate

Underappreciated Angle

The least scrutinized part of Bostrom’s definition is its second clause: “permanently and drastically curtail its potential.” Biological extinction is at least clear-cut, but who decides which futures count as a “curtailment of potential” and which count simply as a different human path? If humanity consciously chose to renounce space colonization or radical biological self-modification, Bostrom’s accounting would classify that restraint as an existential catastrophe, whereas many ethical and religious traditions would call it wisdom. Beneath the appearance of a neutral threat definition, the concept smuggles in a maximalist theory of the human good. That is also why thinkers like John von Neumann and Hans Jonas remain more durable guides here than many contemporary longtermists: both asked how humanity might survive its own technical power without presupposing a single mandatory blueprint for a flourishing future, and Jonas turned uncertainty itself into a method by giving priority to the bad prognosis over the good whenever irreversible stakes are on the table.