Demographic Parity vs. Equal Opportunity: Choosing the Right Test

Opening Scene

Two technicians look at the same set of slides and reach for different stains, because they’re testing for different things: one wants to know whether every sample shows the same overall reading regardless of source, the other wants to know whether every sample that’s genuinely positive gets correctly flagged as positive. Both are legitimate questions. Both can produce a completely different verdict on the exact same batch. Demographic parity and equal opportunity are that same pair of tests, applied to a model’s outputs instead of a set of slides — and knowing which one you actually need to run is half the work.

In Plain English

Demographic parity asks whether different groups receive a positive outcome — a loan approval, a job interview, a favorable diagnosis — at roughly the same rate, regardless of whether that outcome is actually warranted for each individual. Equal opportunity asks a narrower question: among people who genuinely qualify for the positive outcome, are different groups identified at the same rate? These can diverge sharply. A model can satisfy demographic parity while still failing qualified members of one group at a much higher rate than another, and a model can satisfy equal opportunity while still producing very different overall approval rates across groups, if the underlying qualification rates genuinely differ. Choosing between them means deciding which kind of equal treatment actually matters for the decision at hand.

The Old Way

Before demographic parity and equal opportunity were treated as clearly distinct tests:

Teams often used the phrase “equal treatment” loosely, without specifying whether they meant equal outcomes or equal accuracy for those who deserved the outcome.
A model that passed one informal fairness check was sometimes assumed to be fair generally, without anyone testing whether a different, equally valid definition told a different story.
Disagreements about a model’s fairness frequently stemmed from two people unknowingly using two different implicit definitions, talking past each other rather than actually disagreeing on the facts.

Naming these two tests precisely, and knowing they can conflict, is what turns a vague argument into a genuinely productive conversation.

What’s Changing (and Why AI Is the Reason)

Practitioners increasingly specify, explicitly and in writing, which fairness definition a model is being evaluated against, rather than leaving “fair” undefined.
This precision echoes the documentation discipline covered in this content library’s dedicated AI transparency and explainability series, which likewise pushes for specificity over vague reassurance.
As regulators and courts begin referencing specific fairness definitions in guidance and rulings, knowing precisely which test a model satisfies — and which it doesn’t — has become a legal as well as a technical necessity.

The Metaphor, Fully Extended

Two Different Lab Tests	Fairness Concept
Testing whether every sample shows the same overall reading	Demographic parity: equal positive-outcome rates across groups
Testing whether every genuinely positive sample is correctly flagged	Equal opportunity: equal true-positive rates among the qualified
Two valid tests that can disagree on the same batch	Two valid metrics that can disagree on the same model
Choosing the test based on what you’re actually trying to catch	Choosing the metric based on which fairness question actually matters

For Beginners: What to Actually Do

Learn to state, in one sentence each, what demographic parity measures and what equal opportunity measures.
Practice spotting when a fairness claim is ambiguous about which of the two it’s actually referring to.
Get comfortable with the fact that these two metrics can legitimately disagree about the same model.

For Practitioners and Leaders: The Deeper Layer

Decide, before evaluation begins, which fairness definition is appropriate for the specific decision and its stakes, and document that reasoning.
Communicate to stakeholders, in plain terms, which test a model was measured against, and what that test does and doesn’t guarantee.
Watch for cases where satisfying demographic parity would require ignoring genuine, relevant differences in qualification rates, and be prepared to explain that trade-off.

Quick Recap

Demographic parity measures equal outcome rates across groups; equal opportunity measures equal accuracy for the genuinely qualified.
These two metrics can point in opposite directions on the same model.
Vague fairness claims often hide an unstated choice between these two, or similar, definitions.
Choosing and documenting the right test for the decision at hand is a necessary, deliberate step.

Where This Fits in the Series

Article 3 introduced the broader toolkit of fairness metrics. This article zoomed in on the two most commonly confused of them and how to choose between them. Article 5 turns to a related question: before any of these tests can run, which specific attributes actually get tested in the first place.