Harvesting Early

Opening Scene

Article 8 warned against peeking at a trial’s results before the season is over. But there are legitimate reasons to want an early answer — a clearly failing treatment shouldn’t need to run its full planned course before being abandoned, and a clearly successful one shouldn’t need to wait either. The difference between reckless peeking and legitimate early stopping isn’t about whether you look early. It’s about whether your method for looking early accounts statistically for the fact that you’re looking.

In Plain English

Sequential testing methods — including group sequential designs and more modern approaches like always-valid p-values — allow an experimenter to check results at multiple points during a trial and make a legitimate early stopping decision, while mathematically correcting for the increased false-positive risk that repeated looking would otherwise introduce. It’s the rigorous version of the early peek Article 8 warned against.

The Old Way

Before formal sequential testing methods existed, early stopping decisions were either avoided entirely (running every trial to its full pre-planned duration) or made informally, with the inflated false-positive risk Article 8 described:

Clinical trials historically ran to a fixed, pre-planned duration, even when early results made the outcome fairly clear, out of caution about the statistical risks of looking early without proper correction.
Business experiments were often stopped informally the moment results looked favorable, exactly the p-hacking-through-peeking problem Article 8 covered.
Formal statistical correction for repeated looks simply wasn’t widely available or well understood in many applied settings until sequential methods matured.

Sequential testing methods emerged specifically to make legitimate early stopping statistically rigorous, rather than forcing a choice between excessive caution and reckless peeking.

What’s Changing (and Why AI Is the Reason)

Modern experimentation platforms increasingly implement always-valid inference methods by default, letting users check results at any time during a trial without informally inflating the false-positive risk Article 8 warned about.
This has meaningfully reduced the real cost of running experiments — a clearly failing treatment can be abandoned early, freeing up traffic and resources for other tests, without sacrificing statistical rigor.
As organizations run more experiments simultaneously, covered in Article 18, the ability to legitimately stop weak tests early has become a genuine efficiency gain, not just a statistical nicety.

The Metaphor, Fully Extended

The Field Trial	Sequential Testing Concept
Checking a crop’s progress partway through the season, informally	Peeking at results without statistical correction, the trap from Article 8
A formal early-harvest protocol that accounts for the risk of checking early	A sequential testing method that mathematically corrects for repeated looks
Abandoning a clearly failing plot early, freeing resources for other trials	Legitimately stopping a clearly failing test early, freeing traffic for other tests
Confidently declaring an early, clearly successful harvest	Confidently declaring an early, statistically valid significant result

For Beginners: What to Actually Do

Learn the basic difference between informal peeking (Article 8’s trap) and formal sequential testing (this article’s legitimate alternative).
Practice using an always-valid inference method, if your experimentation platform supports one, rather than checking results informally.
Recognize that sequential testing is a genuine statistical discipline, not simply “checking more often” — it requires the correct underlying method.

For Practitioners and Leaders: The Deeper Layer

Invest in experimentation platforms that support formal sequential testing methods, rather than relying on informal, uncorrected peeking.
Use legitimate early stopping to reduce the real cost of running experiments — abandoning weak tests early frees resources for higher-value tests.
Train teams explicitly on the difference between legitimate sequential testing and the false-positive-inflating peeking trap from Article 8.

Quick Recap

Sequential testing allows legitimate early stopping decisions with statistical correction for the increased risk of repeated looks.
It’s the rigorous version of the informal peeking problem covered in Article 8, not the same mistake in disguise.
Modern platforms increasingly implement always-valid inference methods by default.
Legitimate early stopping reduces the real cost of experimentation without sacrificing statistical rigor.

Where This Fits in the Series

Article 15 covered checking a trial early, done rigorously. Article 16 covers what to do when a real field trial genuinely isn’t possible at all.