Model hallucinations

Working Definition

A model hallucination is model-generated content that reads fluently and confidently while lacking support either in the input context or in verifiable facts about the world. Technically, researchers distinguish between an intrinsic hallucination, where the output directly contradicts the source document provided to the model, and an extrinsic hallucination, where the output adds claims that the source neither confirms nor refutes. In everyday usage, the term simply means “something a chatbot made up”: a nonexistent academic paper, a bogus judicial precedent, or an invented quotation. Operationally, one property defines the phenomenon: the error is never flagged as an error, because the model has no built-in threshold below which it refuses to answer.

Genesis and Global Lineage

The word entered machine learning by a circuitous route. In computer vision from the late 1990s onward, “hallucination” meant desirable detail completion, such as reconstructing a high-resolution face from a tiny low-resolution thumbnail (Simon Baker and Takeo Kanade, 1999). In natural language processing, the valence flipped from positive to negative: during the second half of the 2010s, neural machine translation researchers used the term for outputs completely detached from the source sentence, and in 2020 Joshua Maynez and coauthors, studying abstractive summarization, introduced the now-standard distinction between faithfulness to the source document and factual accuracy about the world. Ziwei Ji and colleagues (arXiv 2022; ACM Computing Surveys, 2023) systematized these definitions and metrics just before the term spilled over into mass journalism. The canonical source on our concept list—Emily M. Bender, Timnit Gebru, Angelina McMillan-Major, and Margaret Mitchell’s “On the Dangers of Stochastic Parrots” (2021)—does not analyze hallucinations under that label, but supplies their foundational frame: a language model assembles linguistic forms without access to meaning, while communicative intent and truth value are projected onto the text by human readers. After the launch of ChatGPT in November 2022, general media seized on the term, and Cambridge Dictionary named hallucinate its Word of the Year for 2023. Since then, the concept has lived a double life: as a precise evaluation category inside AI laboratories and as a catch-all popular explanation for every chatbot mistake.

Central European Lineage

We found no documented link between this concept and the intellectual history or historical experience of Central Europe.

Why Now

The phenomenon existed long before ChatGPT, but it became a public crisis only when generative models reached hundreds of millions of users through conversational interfaces lacking any indicator of epistemic uncertainty. In June 2023, Judge P. Kevin Castel of the U.S. District Court for the Southern District of New York fined attorneys $5,000 in Mata v. Avianca for submitting a legal brief citing nonexistent judicial opinions fabricated by ChatGPT; the case became an archetype for dozens of similar incidents across courts, newsrooms, and scientific publishing. In 2024, Sebastian Farquhar and Yarin Gal’s Oxford team published a method in Nature for detecting arbitrary confabulations via semantic entropy, while philosophers Michael Townsen Hicks, James Humphries, and Joe Slater argued that the field should abandon the word “hallucination” altogether in favor of Harry Frankfurt’s concept of “bullshit.” In September 2025, Adam Tauman Kalai, Ofir Nachum, Santosh S. Vempala, and Edwin Zhang shifted the blame from neural architecture to evaluation incentives in “Why Language Models Hallucinate”: models guess because standard benchmarks reward guessing over abstention. The dispute is no longer purely technical; it concerns who bears institutional accountability for confident falsehoods.

Explanatory Power

The concept makes it possible to disentangle three distinct failure modes that everyday talk of “AI making mistakes” collapses into one. First is a knowledge failure: the model never encountered the relevant fact in its training corpus or was trained on noisy data. Second is a faithfulness failure: the model was given an explicit source document in its context window and distorted it anyway—a distinct problem in Retrieval-Augmented Generation (RAG) pipelines, where supplying a verified source does not guarantee a faithful summary. Third is a calibration failure: the model assigns low internal consistency to a claim, yet its decoding and post-training layers produce the same authoritative tone as they do for well-established facts. This three-way distinction has immediate practical consequences. A law firm deploying a drafting assistant must measure citation faithfulness rather than general benchmark accuracy. A newsroom using an LLM to summarize transcripts needs tests for intrinsic hallucinations, because extrinsic fabrications are often easier for editors to spot. And a benchmark designer who scores a blank “I don’t know” at zero—the exact same score as a fabricated answer—actively trains models to gamble, as Kalai and his coauthors demonstrate mathematically.

Axes of Debate

Underappreciated Angle

The least discussed consequence of model hallucinations concerns not the individual user who gets tricked, but the long-term integrity of the documentary record itself. A fabricated citation, date, or biographical detail that slips into a published article, court filing, or encyclopedia entry becomes a real document; that document is scraped into the next web corpus; and the next generation of models learns the fabrication as an attested fact, repeating it with even higher confidence because it now appears across “independent” sources. The problem ceases to be a localized error inside a single neural network and becomes an ecological property of the textual commons, where it grows progressively harder to tell what someone actually verified in an archive from what was merely generated in plausible prose. Projects that require explicit source verification and exact locators—including the knowledge base you are reading—are a direct institutional response to this shift: documentary verification becomes a distinct, mandatory stage of intellectual work once fluency ceases to be a proxy for reliability.