Falsifiability and the replication crisis

Working Definition

Falsifiability is the logical property of an empirical claim that allows it to be refuted by a possible observation: a hypothesis is scientific only if it forbids certain states of affairs and specifies in advance what evidence would prove it wrong. The replication crisis (where replication means repeating a study with new data and reproducibility means regenerating the result from the original data and code) is the discovery, accelerating since 2011, that a large share of published findings in psychology, preclinical medicine, and economics fail to hold up when independent researchers repeat the experiments. Both concepts converge on one operational rule: a finding nobody has tried to refute is not yet established knowledge, only a candidate for knowledge. In casual debate “falsifiability” is often wielded as a club against unwelcome theories; technically, it is a demarcation criterion that tells us whether a claim is testable, not whether it is true.

Genesis and Global Lineage

The criterion of falsifiability was formulated by Karl Popper in Logik der Forschung (1935) against the logical positivists of the Vienna Circle, who grounded scientific meaning in empirical verification. Popper inverted the problem via logical asymmetry: no number of white swans can verify “all swans are white,” whereas one black swan refutes it; science advances by bold conjectures and severe refutation attempts. Seventy years later, John P. A. Ioannidis supplied the second pillar in his 2005 PLoS Medicine paper “Why Most Published Research Findings Are False,” proving via Bayes’s theorem that when researchers test low-prior hypotheses with small samples, analytical flexibility, and publication filters rewarding p < 0.05, most published “discoveries” are false positives. Between 2011 and 2016 that warning became an empirical crisis: Joseph Simmons, Leif Nelson, and Uri Simonsohn (2011) showed how “researcher degrees of freedom” (p-hacking) turn noise into significance; the Open Science Collaboration (2015) repeated 100 psychology studies and replicated barely 36–39 percent; and a 2016 Nature survey found that over 70 percent of researchers had failed to reproduce another scientist’s experiment.

Central European Lineage

Three foundational nodes of this debate lie in Vienna, Lwów, and Debrecen. First, Karl Popper was formed entirely in interwar Vienna: born there in 1902, he studied mathematics, physics, and psychology at the University of Vienna from 1918, earned his doctorate under Karl Bühler in 1928, and wrote Logik der Forschung (Vienna: Julius Springer, 1935 imprint) in debate with the Vienna Circle of Moritz Schlick, Rudolf Carnap, and Otto Neurath. Second, in the same year — 1935 — the Polish-Jewish microbiologist Ludwik Fleck, educated at Jan Kazimierz University in Lwów (now Lviv, Ukraine), former assistant to Rudolf Weigl, and head of his own Lwów laboratory, published Entstehung und Entwicklung einer wissenschaftlichen Tatsache (Genesis and Development of a Scientific Fact), whose manuscript he had sent to Schlick in 1933. Fleck argued that a scientific fact is never a raw observation capable of refuting a theory alone, because perception is shaped by the “thought style” (Denkstil) of a “thought collective” (Denkkollektiv) — a counterpoint that reached global philosophy when Thomas Kuhn credited Fleck in the 1962 preface to The Structure of Scientific Revolutions. Third, the Hungarian philosopher Imre Lakatos — educated in Debrecen (1940–1944, PhD 1947) and Budapest, who first read Popper at the Hungarian Academy of Sciences after surviving the Recsk forced-labor camp (1950–1953) — formulated in London “sophisticated methodological falsificationism” (1970), evaluating whether a “scientific research programme” predicts novel facts or merely patches anomalies.

Why Now

Between 2018 and 2026, the replication crisis evolved from scandal into scientific infrastructure. Following replication projects in psychology, cancer biology (2021), and economics, journals and funders institutionalized preregistration, open data, and Registered Reports, where peer reviewers accept or reject a study before results are known. The COVID-19 pandemic of 2020–2021 dramatized both sides at once: a flood of underpowered preprints claiming miracle cures alongside massive, preregistered platform trials (such as RECOVERY) that rapidly falsified ineffective therapies like hydroxychloroquine. Since 2022, data-fabrication scandals in behavioral research, Alzheimer’s pathology, and oncology — uncovered by independent data sleuths — have shifted debate from statistical errors to academic publishing incentives. Meanwhile, machine-learning pipelines across science since 2023 have triggered a parallel “ML reproducibility crisis,” where subtle data leakage between training and test sets produces illusory breakthroughs.

Explanatory Power

Linking Popperian falsifiability with the replication crisis separates three questions that everyday talk conflates: whether a claim is testable in principle, whether it has been severely tested, and whether that test has been independently repeated. First, it explains why a dramatic headline study with a small sample (n = 40, p = 0.04) carries far less weight than a null result from a preregistered multi-lab study: without high power and a constrained analysis plan, crossing p < 0.05 likely reflects selection noise. Second, Ioannidis’s sixth corollary explains a paradox: the “hotter” and more competitive a field is, the less likely its published findings are to be true, because competition rewards speed and striking positive claims while penalizing slow verification. Third, it clarifies what distinguishes science from its imitations in politics and business: not infallibility, but stating up front what evidence would destroy a claim and paying someone to check.

Axes of Debate

Underappreciated Angle

The replication crisis is usually told as a story about statistics and incentives, rarely as a story about Ludwik Fleck’s sociology of the scientific fact. Yet Fleck in 1935 identified the mechanism behind today’s “garden of forking paths” and experimenter-expectancy effects: a shared thought style determines what counts as an observation, which anomalies are discarded as “equipment error,” and when researchers stop tweaking a protocol because the output “looks right.” Within a tight subfield, a thought collective can spend decades confirming an artifact of its own laboratory routines. Independent replication works not just because it increases sample size N, but because it changes the thought collective: a neutral team, lacking tacit investment in the original lab’s habits, runs the protocol without the micro-adjustments that kept the fragile effect alive. Consequently, while preregistration disciplines the individual researcher, only pluralism of thought styles and adversarial collaboration can test a discipline’s shared blind spots.