Illusion of explanatory depth

Working Definition

The illusion of explanatory depth (IOED) is the systematic gap between how thoroughly people believe they understand the causal workings of a mechanism or phenomenon and how little they can actually articulate when asked to explain it step by step. Operationally, it is measured in three stages: participants rate on a numerical scale how well they understand how a zipper, a flush toilet, or a cap-and-trade emissions system works; write out a step-by-step causal explanation; and rate their understanding again—the drop between the first and second ratings measures the illusion. Often conflated with generic overconfidence or the Dunning–Kruger effect, in Leonid Rozenblit and Frank Keil's technical account it is strictly domain-specific: it afflicts explanatory (causal-mechanistic) knowledge while sparing facts, step-by-step procedures (like baking a cake), and narrative plots, where self-assessments are well calibrated. It is thus a precise failure of metacognitive monitoring in one identifiable class of knowledge.

Genesis and Global Lineage

The term was introduced by Leonid Rozenblit and Frank C. Keil at Yale in their 2002 Cognitive Science paper “The misunderstood limits of folk science: an illusion of explanatory depth” (26(5), pp. 521–562), which across twelve studies demonstrated the post-explanation rating drop and isolated its cognitive drivers. It united two traditions: cognitive work on “intuitive theories” and folk science, which had attributed rich causal models to laypeople, and visual-cognition work on change blindness and the “illusion of perceptual detail,” where observers mistake the ability to re-inspect a visible scene for having stored a detailed mental representation. In a 2003 review in Trends in Cognitive Sciences, Keil embedded the finding in an account of why minds rely on coarse causal pointers backed by a cultural division of cognitive labor. Philip Fernbach and Steven Sloman opened a second branch: in 2013 they carried the paradigm into political psychology (Psychological Science 24(6)), showing that asking voters to explain how a policy works reduced attitude extremity more than asking why they favored it, and in The Knowledge Illusion (2017) they generalized the effect into a theory of the “community of knowledge,” arguing that people routinely mistake knowledge stored in the surrounding social and technological environment for knowledge inside their own heads.

Central European Lineage

The notable Central European connection is a philosophical counterpoint regarding what verbal explanation tests measure. Michael Polanyi—born and educated in Budapest (then Austria-Hungary), where he earned his medical degree in 1913 and his doctorate in physical chemistry in 1917 at the University of Budapest before emigrating to Germany and Britain—summarized his epistemology in The Tacit Dimension (1966) in the maxim that “we can know more than we can tell.” Much of human competence, from clinical diagnosis to operating complex instruments, is tacit and cannot be fully unpacked into explicit propositional steps. Rozenblit and Keil's paradigm measures understanding by a written causal narrative and treats the lower, post-test self-rating as the true baseline. If Polanyi is right, however, part of the drop may reflect the limits of verbalization rather than an illusory grasp of the mechanism: a mechanic or cyclist who stumbles when writing out bicycle physics may still possess genuine functional understanding. Rozenblit and Keil partially anticipate this objection by showing that people calibrate accurately on procedural recipes and that the illusion peaks for multi-level causal systems; nevertheless, the tension between articulability and understanding remains a key boundary condition.

Why Now

Between 2018 and 2026 the concept gained urgency across three domains. First, in misinformation and polarization research: Joseph Vitriol and Jessecae Marsh (2018) linked the illusion to conspiracy beliefs, and Nicholas Light et al. (Science Advances, 2022) showed that opponents of scientific consensus on issues from GMOs to COVID-19 vaccines combine the highest self-rated understanding with the lowest objective knowledge. Meanwhile, preregistered replications by Jarret Crawford and John Ruscio (Psychological Science, 2021) reproduced the drop in perceived understanding but failed to replicate political attitude moderation, and Steven Sloman and Marc Vives (Cognition, 2022) showed that forced explanation softens attitudes only when stances rest on consequentialist outcomes rather than protected moral values. Second, in human–AI interaction: Michael Chromik et al. (2021) demonstrated that local post-hoc explanations of ML decisions (such as SHAP displays) induce an IOED in non-technical users, while Lisa Messeri and M. J. Crockett (Nature, 2024) warned that AI tools foster illusions of understanding in scientists themselves. Third, in cognitive offloading: Matthew Fisher, Mariel Goddu, and Frank Keil's 2015 finding that searching Google inflates ratings of one's own unassisted knowledge became an everyday reality after 2023 as large language models began supplying instant causal explanations on demand.

Explanatory Power

The concept separates having access to an explanation from possessing one inside one's own head. First, in public debates over climate or tax policy, citizens often hold fierce convictions about mechanisms like carbon border adjustments or emissions trading; asking them not why they support or oppose a measure, but how step by step a change in allowance supply alters electricity prices, exposes where causal models end and slogans begin. Second, when an organization deploys a credit-scoring or medical-triage algorithm with an “explainable AI” dashboard, the IOED predicts that compliance officers will feel confident after inspecting feature-attribution charts while remaining unable to predict how the model behaves on an out-of-distribution case—making an active “explain and predict without the dashboard” test far more diagnostic than satisfaction surveys. Third, in science communication, it explains why readers finish an illustrated explainer on mRNA vaccines feeling they grasp the mechanism, yet cannot reconstruct the causal chain ten minutes later. The illusion is strongest wherever visible parts or fluent summaries create the impression that the full mechanism is right at hand.

Axes of Debate

Underappreciated Angle

The most counterintuitive implication of Rozenblit and Keil's experiments concerns the design of information environments: the illusion of explanatory depth is strongest for devices whose outer parts are visible and whose operation seems transparent in real time. That turns conventional wisdom about interface transparency upside down. Frictionless dashboards, interactive visualizations, “Explainable AI” sidebars, and chatbots that generate instant summaries satisfy the exact conditions that maximize the IOED—they inflate the user's feeling of understanding far faster than they build causal competence. Any institution that evaluates risk communication, consent forms, or AI oversight tools by asking users “How well do you understand this?” is mostly measuring the illusion its own smooth interface just created. Genuine comprehension requires built-in cognitive friction—moments where the user must reconstruct or predict the mechanism unassisted—because only stumbling resets our internal gauge of what we actually know.