Regulation Demands a Case File

Opening Scene

A closed case that’s never reviewed by anyone outside the department is one kind of investigation. A case that a court, a regulator, or an outside auditor can demand to see the full file for is another kind entirely — one where the case file itself has to meet a real evidentiary standard, not just satisfy the detective who built it. Explainability has increasingly moved into this second category, driven by real, binding regulation rather than good engineering practice alone.

In Plain English

Several regulatory frameworks now grant individuals a right to explanation for automated decisions that significantly affect them — the EU’s GDPR is the most cited example, but similar principles appear in financial services regulation, employment law, and emerging AI-specific legislation worldwide. This series’ entire toolkit — SHAP, counterfactuals, human review — exists in part specifically to satisfy this real, growing legal requirement, connecting directly to this content library’s dedicated series on data governance and responsible AI.

The Old Way

Before AI-specific regulation existed, similar accountability requirements already governed other consequential automated or semi-automated decisions:

Credit decisions in the United States have long been subject to adverse action notice requirements, requiring lenders to state specific reasons for a rejection, well before modern machine learning existed.
Employment decisions have long been subject to anti-discrimination scrutiny, requiring employers to be able to justify hiring and firing decisions if challenged.
Government agency decisions have long been subject to due process requirements, requiring a reasoned basis that can be reviewed and appealed.

Explainability regulation for AI, in this sense, is often extending existing accountability principles to a new kind of decision-maker, not inventing accountability from scratch.

What’s Changing (and Why AI Is the Reason)

As automated decisions have scaled to affect far more people, far more often than manual human decisions once did, regulators have moved to make existing accountability principles explicit and specific for AI systems, rather than leaving them to be inferred from older, more general law.
Emerging AI-specific regulation, including the EU AI Act and similar frameworks elsewhere, increasingly names explainability and interpretability directly, rather than relying on general-purpose accountability principles alone.
This has turned the entire toolkit this series has covered — from feature importance through counterfactual explanations — from a best practice into, in many jurisdictions and industries, a genuine compliance requirement with real legal consequences for getting it wrong.

The Metaphor, Fully Extended

The Investigation	Regulatory Concept
A case file that must meet a real evidentiary standard	An explanation that must meet a real regulatory standard
An outside body with the authority to demand the file	A regulator with the authority to demand an explanation
Established accountability principles applied to a new context	Existing legal principles extended explicitly to AI decisions
A binding legal requirement, not just good departmental practice	A binding compliance requirement, not just good engineering practice

For Beginners: What to Actually Do

Learn the basic outline of at least one major explainability regulation relevant to your region or industry, such as GDPR’s automated decision-making provisions.
Understand that counterfactual explanations, from Article 7, are often specifically favored by regulators because they’re directly actionable for the affected individual.
Recognize regulatory compliance as a genuine, distinct reason to invest in explainability, alongside the trust and debugging reasons covered earlier in this series.

For Practitioners and Leaders: The Deeper Layer

Map your organization’s model deployments against applicable explainability regulation explicitly, treating gaps as genuine legal and reputational risk.
Connect this work directly to this content library’s dedicated data governance and responsible AI series, which covers the broader compliance landscape this article sits inside.
Build documented, auditable explanation processes now, anticipating that AI-specific regulation will likely keep expanding rather than staying static.

Quick Recap

Several regulatory frameworks, most notably GDPR, grant individuals a right to explanation for automated decisions that significantly affect them.
Similar accountability principles predate AI regulation in credit, employment, and government decision-making.
Emerging AI-specific regulation increasingly names explainability and interpretability directly.
This has turned this series’ entire technical toolkit into a genuine compliance requirement in many contexts, not just a best practice.

Where This Fits in the Series

Article 19 covered the real legal weight now behind explainability. Article 20 closes the series, reassembling the whole investigation into one connected picture.