Data provenance

Working Definition

Data provenance (closely related to data lineage in industrial data engineering) is the documented history of how a specific piece of data came to exist in its current form: from which primary sources it originated, through what sequence of transformations, aggregations, and cleanings it passed, and under which human or institutional agents' responsibility. In its formal computer-science sense (codified in the W3C PROV standard), provenance is a directed graph whose nodes are Entities (data artifacts), Activities (processing steps), and Agents (people, software, organizations), connected by causal and attributive edges such as wasGeneratedBy, wasDerivedFrom, and wasAttributedTo. In everyday parlance the word is often conflated with a simple “source citation,” which misses its core contribution: provenance is not just the initial address, but the entire auditable chain of custody and computation between raw capture and final output. Its diagnostic value lies in letting us interrogate the reliability of a single number, cell, or training token rather than taking an entire database on faith.

Genesis and Global Lineage

The term was borrowed from fine-art connoisseurship and archival science, where provenance denotes either the documented chain of ownership of a painting or the nineteenth-century archival principle (Provenienzprinzip, codified in the 1898 Dutch Manual of Muller, Feith, and Fruin) that records originating from the same institution must not be intermingled with others. It entered theoretical computer science around 2000, when Peter Buneman, Sanjeev Khanna, and Wang-Chiew Tan published Why and Where: A Characterization of Data Provenance (2001), distinguishing between why-provenance (which source rows in a relational database justify the presence of an output row) and where-provenance (from which exact source cell a value was copied)—soon completed by how-provenance (algebraic semirings capturing how source tuples combined to produce a result). By the mid-2000s, computational science (e-science) adopted the concept to track complex bioinformatics and physics workflows (surveyed by Simmhan, Plale, and Gannon in 2005). A series of international “Provenance Challenges” led by Luc Moreau produced the Open Provenance Model and culminated on April 30, 2013, in the W3C PROV family of standards. Around 2018, the concept was drafted into two urgent public arenas: machine-learning ethics (Timnit Gebru et al.'s Datasheets for Datasets, 2018/2021) and digital media authentication, where the C2PA coalition uses content provenance for cryptographically signed camera and editing metadata.

Central European Lineage

We found no documented link between this concept and the intellectual history or historical experience of Central Europe.

Why Now

Between 2018 and 2026, three developments turned data provenance from a quiet database specialty into a front-line legal and engineering battleground. First, generative AI models were trained on massive web scrapes stripped of licensing and origin metadata; in October 2023, the Data Provenance Initiative (Longpre, Mahari et al.) audited over 1,800 fine-tuning datasets and discovered that more than 70% had omitted or misstated licenses on platforms like Hugging Face, while high-stakes copyright lawsuits (such as The New York Times v. OpenAI, filed in December 2023) hinge directly on proving which exact training works entered a model's corpus and whether synthetic training data will trigger “model collapse.” Second, the European Union's AI Act (enacted in 2024), building on GDPR (RODO in Poland) data-governance rules, explicitly mandates in Article 10 that providers of high-risk AI systems document “data collection processes and the origin of data,” while requiring general-purpose AI providers to publish sufficiently detailed summaries of training content. Third, the explosion of photorealistic image, audio, and video generators pushed provenance onto consumer screens: the Coalition for Content Provenance and Authenticity (C2PA)—backed by Adobe, Sony, Canon, Microsoft, OpenAI, and the BBC—rolled out Content Credentials, shifting verification from hopeless post-hoc deepfake detection to cryptographic origin signing at the moment a camera shutter clicks.

Explanatory Power

The concept demonstrates that “data quality” is not an intrinsic property of a static file, but a historical property of the pipeline that produced it—and that many intractable disputes over “facts” are actually disputes over undocumented transformations. Three concrete examples illustrate this. First, when Stanford researchers and journalists discovered in 2023–2024 that LAION-5B, the benchmark image-text dataset used to train popular diffusion models, contained thousands of links to illegal child sexual abuse material and private medical photographs, the root failure was a total absence of provenance filtering in bulk Common Crawl scraping: no one could trace or purge downstream derivatives without retraining from scratch. Second, in courtroom forensics, digital evidence is admissible not because a video “looks convincing,” but because a documented chain of custody links the physical recording device via cryptographic hashes to the exhibit presented to the judge—which is W3C PROV dressed in legal robes. Third, in corporate and public analytics, when two ministry departments or business units present conflicting unemployment or active-customer totals at a board meeting, neither SQL query is usually “wrong”: they diverge because one pipeline silently dropped inactive accounts after 90 days while the other used 180 days, and only fine-grained column-level data lineage can resolve the dispute without bureaucratic trench warfare.

Axes of Debate

Underappreciated Angle

Beneath its dry engineering surface, data provenance is the formal computer-science incarnation of the epistemology of testimony. For centuries, philosophers from David Hume and Thomas Reid to contemporary social epistemologists have asked under what conditions we are justified in believing a claim we did not witness firsthand. The W3C PROV model gives a structural answer: epistemic warrant does not reside inside a proposition (or a number in a spreadsheet), but in the topology of the graph connecting agents and transformations—for instance, whether two seemingly independent confirmations actually trace back to the very same upstream scrape (a hidden single point of failure that journalists call “circular reporting” and intelligence analysts call “incestuous amplification”). Yet this reveals a fundamental thermodynamic-style paradox of metadata: fine-grained how-provenance grows faster than the data themselves. If you record every intermediate floating-point operation, your provenance log quickly dwarfs your database; if you summarize, you reintroduce the very opacity provenance was meant to cure. Practical provenance is therefore never a neutral mirror of the past, but an economic bet on which future accountability questions—from a court, a regulator, a peer reviewer, or a debugging engineer—you will eventually be forced to answer.