🌱

Experimentation & A/B Testing

Proving an idea works before betting the business on it.

Part 1

Two Plots, One Season

why comparing a fertilized field to last year's unfertilized field proves almost nothing, and what that reveals about the whole point of a controlled experiment.

Part 2

What the Agronomist Is Actually Testing

the real difference between noticing a pattern in your data and rigorously proving that one specific change caused it.

Part 3

Judging by a Good-Looking Season

why relying on gut feel and a single good result is exactly the trap randomized experimentation was invented to escape.

Part 4

Randomizing the Plots

why randomly assigning which plot gets the treatment is the single most important design decision in any real experiment.

Part 5

The Control Plot

why every real experiment needs a group that gets no change at all, and why 'no change' is a genuinely informative baseline, not a wasted opportunity.

Part 6

One Variable at a Time

why changing the fertilizer, the seed, and the watering schedule all at once makes it impossible to know which change actually mattered.

Part 7

How Big Does the Field Need to Be

why testing a new fertilizer on three plants tells you almost nothing, and how statistical power determines the minimum trial size that's actually worth running.

Part 8

The Season Wasn't Long Enough

why checking a trial's results too early, or ending it too soon, produces misleading conclusions even with a perfectly designed experiment.

Part 9

A Rogue Weed in the Data

how a bug in the assignment logic, a broken tracking pixel, or a mislabeled group can silently invalidate an otherwise well-designed experiment.

Part 10

Reading the Yield

what a p-value actually means, and the specific, narrow claim it does and doesn't support.

Part 11

A Difference That Doesn't Matter

why a statistically significant yield increase of half a gram per plant might not be worth the cost of the new fertilizer at all.

Part 12

Testing More Than One Fertilizer

how multivariate testing lets an experimenter test several changes at once without sacrificing the ability to isolate each one's real effect.

Part 13

The Neighboring Field Effect

why a treatment applied to one plot can quietly affect its untreated neighbor, silently violating one of experimentation's core assumptions.

Part 14

Different Soil, Different Result

why a fertilizer's average effect across the whole farm can hide the fact that it helps sandy soil and actually hurts clay soil.

Part 15

Harvesting Early

how sequential testing methods let an experimenter check results early and stop legitimately, without inflating the risk of a false conclusion.

Part 16

When You Can't Split the Field

what to do when randomization genuinely isn't possible, and why a quasi-experiment is a real, useful fallback rather than a lesser substitute.

Part 17

The Automated Greenhouse

how modern experimentation platforms automate randomization, power calculations, and analysis, turning rigorous testing into routine infrastructure.

Part 18

Too Many Trials at Once

why running a hundred simultaneous trials guarantees several will look significant by pure chance alone, and what to do about it.

Part 19

From the Test Plot to the Whole Farm

why a winning result on a small test plot doesn't automatically guarantee the same result once it's rolled out to the entire farm.

Part 20

The Complete Growing Season

reassembling the whole discipline, from two mismatched plots to a fully validated result rolled out safely across the entire farm.