Too Many Extras, Not Enough Story

Opening Scene

A production that leans too heavily on digitally generated crowds, over and over, across many films, risks something subtle but real: the generated crowds start to look like echoes of previous generated crowds rather than genuine reflections of how real people actually look and move. Each generation drifts a little further from the original, real reference footage that grounded the very first version. Machine learning has recently discovered a strikingly similar risk, with a specific name: model collapse.

In Plain English

Model collapse is a documented phenomenon where models trained repeatedly, generation after generation, substantially on synthetic data generated by previous models, gradually lose diversity and drift away from the true underlying real-world data distribution. Each generation’s errors and simplifications compound, and rare, unusual, or “tail” patterns that appeared in the original real data can disappear from the training distribution entirely after several generations of synthetic-on-synthetic training.

The Old Way

Before model collapse was formally studied and named, the underlying risk of “copies of copies” degrading was already a familiar phenomenon in other contexts:

Photocopying a photocopy repeatedly produces a visibly degraded image, with fine detail lost a little more at each generation — a simple, physical analog of the same underlying risk.
Oral traditions passed down through many retellings were long known to drift from their original details, each retelling introducing small, compounding changes.
Generation after generation of imitation in any creative field was long understood to risk drifting from an original, grounding source, without a new infusion of fresh, original material.

The specific machine learning phenomenon is new; the underlying pattern of degradation through repeated self-reference is not.

What’s Changing (and Why AI Is the Reason)

As LLM-generated text and AI-generated images have become increasingly common across the internet, the risk that future models will be trained substantially on past models’ synthetic output — even unintentionally — has grown into a genuine, active research concern.
Researchers have demonstrated model collapse empirically in controlled studies, showing measurable diversity loss and distributional drift across successive generations of synthetic-on-synthetic training.
This has reinforced the practical importance of maintaining genuine, verified real data sources, and of the fidelity evaluation and quality assurance practices covered in Articles 12 and 15, as defenses against unknowingly training on excessive amounts of synthetic content.

The Metaphor, Fully Extended

The Film Set	Model Collapse Concept
Digitally generated crowds echoing previous generated crowds, not real footage	Models trained on previous models’ synthetic output, not real data
Each generation drifting further from the original real reference	Each training generation drifting further from the true real-world data distribution
Fine, unusual details disappearing after repeated photocopying	Rare, tail patterns disappearing after repeated synthetic-on-synthetic training
A production that keeps a genuine, real reference to ground every generated shot	A training pipeline that maintains genuine, verified real data as an anchor

For Beginners: What to Actually Do

Study the basic mechanism behind model collapse — compounding errors and diversity loss across generations — to understand why this risk is genuine and not merely theoretical.
Practice tracking what proportion of any training dataset is synthetic versus genuinely real, especially for text and image data scraped broadly from public sources.
Recognize this as an active, evolving research area, worth staying current on rather than treating as fully settled.

For Practitioners and Leaders: The Deeper Layer

Maintain and protect access to genuine, verified real data sources as a defense against model collapse, even as synthetic data use grows.
Apply the fidelity evaluation and quality assurance practices from Articles 12 and 15 specifically to guard against unknowingly training on excessive synthetic content.
Track the proportion of synthetic-versus-real data in training pipelines explicitly, particularly for models trained on broad, internet-scraped sources where this ratio can shift without anyone noticing.

Quick Recap

Model collapse is a documented phenomenon where models trained repeatedly on synthetic output from previous models lose diversity and drift from the true data distribution.
Rare, unusual patterns from the original real data can disappear entirely after several generations of synthetic-on-synthetic training.
This risk has grown as AI-generated content has become increasingly common across the internet.
Maintaining genuine, verified real data sources is the primary defense against this risk.

Where This Fits in the Series

Article 16 covered the risk of leaning too heavily on generated content over successive generations. Article 17 covers a different, more deliberate risk: synthetic media used to actively deceive.