When the Supplier Changes the Recipe

Opening Scene

A supplier has delivered the same cut of beef every week for two years, no discussion needed. One week, without telling anyone, they swap in a slightly different cut that looks almost the same in the crate but cooks completely differently. Nobody notices until the dish comes out wrong, table after table, before anyone in the kitchen realizes the ingredient itself changed, not the cooking.

That’s schema drift — a source quietly changing shape without announcing it — and the real question isn’t just how to detect it. It’s how a kitchen keeps running safely when it happens anyway.

In Plain English

Schema drift is when a data source changes its structure — a renamed field, a new column, a data type that quietly shifts, a value that used to always be present and now sometimes isn’t — without any formal warning to the systems depending on it. Article 2 in this series covered spotting drift at the point of extraction. This article is about what a pipeline actually does once drift happens: whether it fails safely, adapts automatically, or — worst case — keeps running and quietly produces wrong results.

The Old Way

Traditionally, a pipeline’s response to schema drift was determined entirely by how it happened to be written, not by any deliberate design choice. Some pipelines were brittle in a useful way: a missing expected column made them crash loudly, which was annoying but at least visible — the kitchen equivalent of the whole line stopping the moment someone notices the beef looks wrong. Others were brittle in a dangerous way: they kept running, silently treating a changed field as if nothing had happened, and produced confidently wrong output with no warning at all.

Neither was actually designed behavior in most cases — it was just whatever the original code happened to do when it hit something unexpected, discovered the hard way, usually during an incident.

What’s Changing (and Why AI Is the Reason)

1. Drift gets caught before it causes damage, more often. As covered in Article 2, AI-assisted monitoring can now flag a source that looks different than usual before that difference propagates through transformation and load — closer to someone at the delivery dock catching the ingredient swap before it ever reaches the stove.
2. Pipelines can adapt to some kinds of drift automatically. For well-understood, low-risk changes — a new optional field appearing, a column being renamed in an obviously equivalent way — AI-assisted tooling can increasingly propose or apply a compatible adjustment automatically, rather than every drift event requiring a person to intervene before anything can run again.
3. The line between “safe to auto-adapt” and “needs a human” is getting clearer. Not all drift is equal — a new optional field is very different from a currency field silently switching units. AI tools are increasingly able to help classify which kind of change just happened, so the response (auto-adapt, alert-and-pause, fail loudly) can match the actual risk, rather than treating every drift event the same way.

The Metaphor, Fully Extended

Kitchen Element	Schema Drift Response Concept
A supplier quietly swapping an ingredient	A source changing its schema without warning
The line stopping the moment something looks wrong	A pipeline failing loudly on unexpected structure
Cooking with the swapped ingredient as if nothing changed	A pipeline silently processing drifted data as if it were normal
A kitchen manager catching the swap at the delivery dock	Drift detected at extraction, before it propagates
Automatically adjusting a recipe for a clearly equivalent substitute	A pipeline auto-adapting to a low-risk, well-understood schema change
Refusing to guess when a swap changes something that actually matters	A pipeline pausing and alerting on high-risk, ambiguous drift

Quick Recap

• Schema drift is a source changing shape without warning; this article is about surviving it, not just detecting it.
• Traditionally, a pipeline’s drift behavior was accidental — determined by how the code happened to be written, not deliberate design.
• AI tools can increasingly classify drift by risk and safely auto-adapt to low-risk changes while flagging higher-risk ones for a person.
• Silent, unexamined continuation under drift is the most dangerous default, because nothing signals a problem exists.
• Deliberately test your pipelines against broken input to learn their real behavior under drift, rather than assuming you know it.

Where This Fits in the Series

Article 11 covered catching mistakes before they run. This article covered surviving a specific kind of surprise once it happens anyway. Article 13 looks at a different way of watching the source altogether — catching changes the moment they happen at the source system itself, not after the next scheduled run.