Foundation models

Working Definition

A foundation model is a machine-learning model trained on broad data at massive scale—typically via self-supervision—that is not built for a single narrow application, but serves as a shared base to be adapted (through fine-tuning, in-context prompting, or post-training layers) across a wide variety of downstream tasks. Crucially, the term describes a model’s architectural and economic position in a software supply chain rather than a specific neural architecture or parameter count: a single model becomes “foundational” when hundreds of distinct medical, legal, coding, or consumer systems are built atop it, whereas a similarly large model trained solely for weather forecasting or protein folding is not. In casual usage, “foundation model” is often treated as a synonym for a large language model (LLM); in its precise technical sense it spans text, vision, audio, robotics, and multimodal systems, while in European Union law (the 2024 AI Act) its regulatory counterpart is codified as a “general-purpose AI (GPAI) model.”

Genesis and Global Lineage

The term was coined in August 2021 in the 214-page report On the Opportunities and Risks of Foundation Models, co-authored by 114 Stanford researchers led by Rishi Bommasani and Percy Liang to launch the Center for Research on Foundation Models (CRFM). The report did not invent a new algorithm: transfer learning, ELMo, BERT (2018), and GPT-3 (2020) already existed. Instead, the Stanford team named an institutional and epistemic shift defined by two properties: emergence (capabilities implicitly induced by scale rather than explicitly programmed) and homogenization (the consolidation of the AI field around a handful of shared base models, so downstream systems inherit the same capabilities and blind spots). The authors rejected existing labels: “large language model” excluded vision and multimodal systems; “self-supervised model” described a training objective rather than downstream use; and “pretrained model” implied the important work happens afterward. They chose the noun adjunct foundation over foundational to convey an architectural substructure that is indispensable yet unfinished—a base whose cracks threaten the entire building. Within two years, the term entered statutory law: U.S. Executive Order 14110 (October 2023) defined “dual-use foundation models,” while the EU AI Act (2024) adopted “general-purpose AI model” after intense lobbying over upstream versus downstream liability.

Central European Lineage

When the 2021 Stanford CRFM report reconstructs the technical genealogy of self-supervised pretraining (Section 1.1), it traces the paradigm “dating back to word embeddings” and explicitly cites the 2013 word2vec paper by Tomáš Mikolov and colleagues. Mikolov received his entire academic formation in Central Europe, at the Faculty of Information Technology of the Brno University of Technology (Vysoké učení technické v Brně, VUT) in Czechia. Working in Jan Černocký’s speech group in Brno, Mikolov published Recurrent neural network based language model at Interspeech 2010 and defended his PhD dissertation, Statistical Language Models Based on Neural Networks, in October 2012, before moving to Google where word2vec was completed months later. Neither the socio-technical coinage “foundation model” nor the Stanford authors come from the region, but the initial self-supervised representation-learning step they identify as the start of the lineage grew directly out of the Brno school of neural speech modeling.

Why Now

Between 2020 and 2026, foundation models moved from a research pattern to a contested layer of global digital infrastructure. GPT-3 (2020) showed that a single pretrained model could be adapted via prompts; Stanford CRFM (2021) named the paradigm; and ChatGPT (2022) and GPT-4 (2023) turned upstream model APIs into a utility layer for millions of businesses. Regulation followed: during late-2023 trilogue negotiations on the EU AI Act (Regulation 2024/1689), rules for “foundation models” (termed “general-purpose AI models” in Articles 51–55) nearly derailed the law as France, Germany, and Italy sought to shield European startups like Mistral and Aleph Alpha. The final compromise created a two-tier regime: baseline transparency and copyright-summary obligations for all GPAI models, plus stringent risk-mitigation duties for models presumed to pose “systemic risk” above 10²⁵ training FLOPs. Simultaneously, open-weight models—Meta’s Llama, Mistral, and China’s Qwen and DeepSeek (2024–2025)—ignited debates over sovereign AI infrastructure, spurring domestic initiatives in Central Europe such as Poland’s open-source Bielik (2024) and state-backed PLLuM (2024–2025).

Explanatory Power

The concept makes visible supply-chain dependencies that disappear when analysts look only at end-user applications. First, it explains the downstream cascade of defects (homogenization of failure): if five hundred hospital triage tools, banking chatbots, and welfare assistants call the same foundation model, a single latent bias, hallucination, or prompt-injection vulnerability in that base model propagates across all five hundred institutions—meaning that auditing a downstream app without access to upstream training data and weights is regulatory theater. Second, it clarifies AI’s political economy: because pretraining requires hundreds of millions of dollars in GPUs and energy while adaptation is cheap, the market gravitates toward an oligopoly of upstream “model landlords” collecting API rent from thin application wrappers. Third, it maps legal responsibility when three distinct actors—upstream model developer, midstream fine-tuner, and downstream deployer—contribute to a harmful decision.

Axes of Debate

Underappreciated Angle

The most profound shift captured by the concept is epistemological: it quietly moved the primary unit of computer science from the algorithm to the artifact. For half a century, machine learning operated like a mathematical science: a lab published a method (SVMs, backpropagation, LSTM, ResNet), and any university lab could re-implement the equations from scratch. A foundation model, by contrast, is a singular historical artifact—a multi-gigabyte matrix of weights produced by a one-off industrial run costing tens or hundreds of millions of dollars on a proprietary data mixture no outsider can inspect or replicate. Consequently, most academic “AI research” since 2021 no longer consists of training new learning algorithms from scratch, but of probing, benchmarking, or fine-tuning corporate artifacts whose internal formation remains opaque. Artificial intelligence has begun to resemble big-instrument sciences like particle physics—except that the “telescopes” (GPT, Claude, Gemini, Llama) are privately owned commercial products rather than public consortia like CERN.