Which Farm Did This Carrot Come From

Opening Scene

A customer complains that a dish tasted off. A kitchen with good records can trace it back precisely: this dish, made from this batch of prepped vegetables, which came from this delivery, from this specific farm, received on this exact morning. A kitchen without those records can only guess — check today’s deliveries, question the whole prep team, maybe never actually figure out what happened.

That traceability — being able to follow a finished dish all the way back to its original ingredients — is what data lineage gives a pipeline, and it matters most in exactly the moment something’s gone wrong.

In Plain English

Data lineage is the record of where a piece of data came from and every transformation it passed through on the way to where it ended up. When a number on a dashboard looks wrong, lineage is what lets someone trace it backward — this metric, calculated from this transformation, which combined these two source tables, which were extracted from these two systems — instead of guessing at every step along the way.

The Old Way

Traditionally, lineage existed only in the heads of the people who built a pipeline, if it existed anywhere at all. A senior engineer might know, from memory, that a particular report’s revenue figure ultimately traced back through four transformation steps to two source systems — but that knowledge left the building whenever they did. Documentation, when it existed, was usually written once and stale within months, the data equivalent of a kitchen’s delivery logs that stopped being updated the moment things got busy.

Without reliable lineage, debugging a wrong number meant manually tracing backward through code, one transformation at a time, often across a system built by people who’d since left — slow, uncertain work that depended entirely on whether anyone still around understood the pipeline well enough to reconstruct its history.

What’s Changing (and Why AI Is the Reason)

1. Lineage can be inferred automatically, not just documented by hand. AI-assisted tooling can now analyze a pipeline’s actual code and query patterns to reconstruct lineage automatically — building the delivery log after the fact, from the kitchen’s actual behavior, rather than relying on someone having written it down in advance.
2. Tracing a bad number backward is faster. When something looks wrong, AI tools can help walk the lineage graph and surface the most likely point of failure — the specific transformation or source most associated with the anomaly — instead of a person manually checking each step in sequence.
3. Lineage is becoming a requirement for AI agents, not just human auditors. An AI agent that’s asked to explain or justify a number it’s reporting increasingly needs to cite where that number actually came from — lineage isn’t just documentation for a human debugging session anymore, it’s an input the agent itself may need to answer a question credibly.

The Metaphor, Fully Extended

Kitchen Element	Data Lineage Concept
The delivery log showing which farm sent which crate	Records of a data source’s origin
Tracing a dish back through prep to a specific delivery	Tracing a transformed value back through pipeline steps
A senior chef who remembers every supplier from memory	Lineage that exists only as undocumented institutional knowledge
Delivery logs that stopped being updated once things got busy	Documentation that goes stale shortly after a pipeline is built
Reconstructing what happened from receipts and prep notes after the fact	AI-inferred lineage built from a pipeline’s actual code and behavior
A health inspector asking for a dish’s full ingredient trail	An AI agent needing to justify where a reported number came from

Quick Recap

• Data lineage traces a piece of data back through every transformation to its original source — essential for debugging and for trust.
• Traditionally, lineage lived mostly in people’s memory and stale documentation, both unreliable over time.
• AI tools can now infer lineage automatically from a pipeline’s actual code and behavior, and help trace a bad number back to its likely cause faster.
• AI agents increasingly need lineage themselves, to credibly justify the numbers they report.
• A useful test: can your team trace an important number back to its source quickly? If not, that’s worth fixing before you need it during an actual incident.

Where This Fits in the Series

Article 9 covered what happens to records that fail along the way. This article covered tracing the ones that succeed, all the way back to where they started. Article 11 looks at how a pipeline earns the right to be trusted before it ever runs in production — the discipline of actually testing it.