Fact-Checking a Data Story Before It Ships

Opening Scene

A major feature clears its editor, reads well, has a strong lede and a clean structure — and still goes to a fact-checker before it runs. The fact-checker doesn’t care whether the story is well told. Their entire job is narrower and more mechanical: trace every specific claim back to its source, confirm every number, every quote, every attributed fact, independently of whether the surrounding prose is any good. A beautifully written story built on one unverified number is still a story that shouldn’t run. Fact-checking and editing are deliberately different jobs, done by different eyes, because a good editor reading for narrative flow is not the same skill as a fact-checker reading for verifiability, and conflating the two lets errors slip through in the gap between them.

Data narratives need this exact second, narrower pass — and it’s easy to assume the editorial review from Article 10 already covers it, when it usually doesn’t.

In Plain English

Fact-checking a data story means independently verifying every specific, checkable claim it makes — every number, every comparison, every stated trend — against its actual underlying source, separately from any review of whether the story is well structured or well written. This includes checking things that are easy to get subtly wrong even in a well-intentioned report: that a percentage change was calculated against the right baseline, that a comparison is apples-to-apples, that a chart’s axis or scale doesn’t visually exaggerate a real but modest difference, and that no claim has outrun what the underlying data can actually support.

The Old Way

Without a dedicated fact-checking pass, data narratives tend to rely on one of these weaker substitutes:

Assuming editorial review covers it — treating narrative review, which checks structure and framing, as if it also verifies underlying accuracy, when in practice a reviewer focused on flow and clarity is not reliably also checking every number against its source.
Trusting the pipeline — assuming that because the data pipeline is well tested, every claim derived from it is automatically correct, when many errors happen in the interpretive step between a correct number and the sentence describing it, not in the pipeline itself.
Spot-checking only the headline number — verifying the one figure that leads the piece while leaving supporting comparisons, secondary stats, and chart scaling unchecked, on the assumption that the most prominent claim is the only one worth the effort.

Each of these leaves real gaps, because narrative quality, pipeline correctness, and headline accuracy are each necessary but none of them, alone or together, guarantee that every claim in a piece is actually verifiable and correctly stated.

What’s Changing (and Why AI Is the Reason)

AI-generated narratives can introduce a specific new failure mode: a claim that sounds precisely sourced but has been subtly paraphrased away from what the data actually supports. A model summarizing “conversion improved in most regions” from data where it improved in a bare majority, or rounding a comparison in a way that changes its meaning, produces text that reads as carefully fact-based without necessarily being checked against the source at the level of precision the wording implies.
AI tools can now assist fact-checking itself, by tracing a stated claim back to the specific query or data slice it should match and flagging mismatches automatically. This is a genuine acceleration of a traditionally slow, manual task, but it still requires a human to define what “matches” means for a given claim and to review any flagged discrepancy for actual significance.
The growing fluency and volume of AI-generated narrative text has made fact-checking a more urgent gate than it was when every report was manually written and inherently rate-limited by analyst time. A well-written but unverified claim can now be produced and distributed far faster than a human fact-checking process can naturally keep pace with, unless that process is deliberately resourced to match.

The Metaphor, Fully Extended

Newsroom Element	Data Storytelling Concept
A fact-checker tracing every claim back to its source, independent of the editor	Verifying every specific claim in a report against its underlying data, separate from narrative review
A beautifully written story built on one unverified number, held from running	A well-structured report containing one unverified or miscalculated claim
Checking that a quote was actually said, not just that it reads well	Checking that a percentage was calculated against the correct baseline, not just that it sounds plausible
Assuming the copy editor already checked the facts	Assuming narrative/editorial review already covers underlying data accuracy
A fact-checking assistant tracing a claim to its source query and flagging mismatches	AI tools assisting fact-checking by tracing stated claims back to their underlying data slice

For Beginners: What to Actually Do

Treat fact-checking as a distinct step from proofreading or narrative review — do it separately, and specifically trace every number back to its source query or table.
Pay particular attention to comparisons and percentage changes, since these are where baseline errors and subtly misleading calculations most often hide.
Check chart scales and axes specifically for whether they visually exaggerate or understate the real magnitude of a difference.
When using an AI-assisted fact-checking tool, review any flagged discrepancy yourself rather than assuming a “no issues found” result means every claim was actually checked with sufficient precision.

For Practitioners and Leaders: The Deeper Layer

Establish fact-checking as a formally separate step from editorial narrative review, with its own owner and its own checklist, rather than assuming one review covers both.
Invest in AI-assisted claim-tracing tools to keep fact-checking capacity in line with growing report volume, while keeping a human in the loop for judging the significance of any flagged mismatch.
Set an explicit policy for which claims require full source verification versus which can rely on pipeline correctness alone, based on how consequential the claim is.
Recognize that AI-generated narrative introduces new, subtle failure modes — like paraphrasing that quietly shifts a claim’s meaning — that specifically warrant fact-checking attention beyond what manually written reports historically needed.

Quick Recap

Fact-checking independently verifies every specific claim in a data story against its underlying source, as a distinct step from editorial review of structure and framing.
Without it, teams rely on weaker substitutes: assuming editorial review covers accuracy, trusting the pipeline blindly, or spot-checking only the headline number.
AI-generated narrative can introduce subtle paraphrasing errors that sound precise without having been checked at that level of precision.
AI can accelerate fact-checking by tracing claims to their source, but judging the significance of any discrepancy remains a human task.

Where This Fits in the Series

Article 10 established the editorial review process broadly; this article isolates fact-checking as its own necessary, narrower discipline within it. Article 12 looks at keeping narrative quality consistent across a whole team, not just correct within any single report.