Information hazards

Working Definition

An information hazard is a risk that arises from the dissemination—or potential dissemination—of true information: knowledge that can cause harm directly or enable an agent to cause harm. Operationally, the question is not “is this true?” but “who, upon learning this, will gain the capability or motive to cause harm, and on what scale?” In Nick Bostrom’s technical formulation, the concept is a taxonomy of roughly thirty distinct risk modes, ranging from a pathogen’s genome sequence to psychological reactions in the recipient; in everyday internet usage, the slang contraction infohazard (coined on rationalist forums such as LessWrong) means any thought one is better off not knowing, often used half-jokingly. Crucially, the concept excludes false information: disinformation and propaganda are separate problems.

Genesis and Global Lineage

The term and its typology were formulated by Nick Bostrom in his article “Information Hazards: A Typology of Potential Harms from Knowledge” (Review of Contemporary Philosophy, vol. 10, 2011, pp. 44–79), written at Oxford within the existential-risk research program. Bostrom’s starting point is the observation that scientific culture operates with a strong default presumption in favor of openness, treating exceptions—military secrecy, patient confidentiality, double-blind trials—as isolated special cases lacking a shared analytical vocabulary. Bostrom distinguishes modes of transmission (data, idea, attention, template, signaling, evocation) and domains of harm (adversary advantage, markets and organizations, human psychology, information systems, technological development). The concept inherits from the physicists’ debates over secrecy between 1939 and 1945 and the post-2001 controversies over dual-use research of concern in biology, but Bostrom was the first to organize them into a unified catalog. In the 2010s, the term was captured by online subcultures, where infohazard came to signify a memetic curse rather than a quantifiable policy risk; the two registers need to be read apart.

Central European Lineage

Decades before Bostrom coined the term, the Hungarian-born physicist Leó Szilárd—best known to English-speaking readers for the Manhattan Project, but shaped in his youth by Budapest’s scientific culture—enacted in practice the exact distinction Bostrom later called idea hazard and data hazard. Szilárd began engineering studies at the Technical University of Budapest in 1916, interrupted them for service in the Austro-Hungarian army, and left for Berlin in December 1919, where he switched to physics. The idea of a nuclear chain reaction struck him in London in 1933; he filed a patent in 1934 and assigned it to the British Admiralty so that it would remain classified, keeping its contents out of public view until 1949. In 1939, after the discovery of uranium fission, he urged Enrico Fermi in New York and Frédéric Joliot-Curie in Paris to refrain from publishing their measurements of secondary neutrons; Fermi reluctantly agreed, while Joliot refused and his team published the neutron numbers in Nature in April 1939. Bostrom mentions Szilárd in a single footnote sentence as proof that the physicists’ naïveté was avoidable “at least as early as 1933,” citing Richard Rhodes. This is not a direct intellectual influence on Bostrom’s taxonomy, but an earlier, documented practice of the same reasoning—though the regional link is tenuous, since Budapest gave Szilárd his initial engineering training rather than his physics or the London insight itself.

Why Now

In February 2019, OpenAI declined to release the full version of its GPT-2 language model, citing misuse risks, and in its report Release Strategies and the Social Impacts of Language Models (August 2019, expanded in November) justified staged release; from that moment on, information-hazard calculus became routine across frontier AI laboratories, from system cards to biological and chemical capability thresholds in responsible-scaling policies. That same year, Gregory Lewis and co-authors published “Information Hazards in Biotechnology” in Risk Analysis, written in the wake of disputes over publishing gain-of-function experiments on H5N1 influenza transmission (2011–2012) and the synthesis of horsepox virus (2017). The 2020 COVID-19 pandemic and the debate over the origins of SARS-CoV-2 demonstrated that merely discussing what is technically feasible in a virology lab can act as an attention hazard. Simultaneously, generative AI models have lowered the cost of moving from an abstract idea to actionable data: what in 2011 was the gap between “knowing something is possible” and “having the step-by-step recipe” has by 2025 narrowed to a single well-crafted prompt.

Explanatory Power

The concept makes visible that “true” and “safe to broadcast” are two orthogonal axes rather than one, and that scientific institutions are built to measure only the first. Three applications illustrate its reach. First, attention hazard explains why detailed reports on security vulnerabilities—in computer networks, biosecurity, or critical infrastructure protection—can cause net harm even when every single fact is already scattered across public literature: an attacker often needs not new data, but a signpost showing where to look. Second, the distinction between idea hazard and data hazard explains why proving that a feat is possible (a nuclear test, a proof-of-concept exploit) is sometimes more dangerous than leaking blueprints: knowing that a path works drastically shortens a rival’s search process. Third, the category of psychological hazards (knowledge of a terminal diagnosis, of catastrophe probabilities, or of others’ candid opinions) shows that the medical “right not to know” and the secret ballot belong to the same family of institutional decisions as classifying pathogen sequences.

Axes of Debate

Underappreciated Angle

The least discussed part of Bostrom’s taxonomy is the attention hazard, because it destabilizes the very act of writing about risks. Every biosecurity threat assessment, every catalog of model jailbreaks, and every analytical briefing sorts the space of possibilities by effectiveness for a potential attacker. This creates a paradox that Bostrom notes but does not resolve: the more thorough our defense, the more its documentation reads like an attack map. In practice, this pushes fields toward an esoteric culture in which the most serious risk assessments circulate only within closed circles while public debate receives a sanitized, vague version. That secrecy carries an epistemic cost that boomerangs on regulators: a public kept in the dark about the concrete arguments cannot judge whether classification is justified, and reacts with either panic or dismissal.