Fairness Testing Guide

How to audit ML models for bias, with practical code examples and decision frameworks.

Why Fairness Testing Is Non-Negotiable

A model can have 99% accuracy and still be discriminatory. If it learned historical biases from training data (and it almost certainly did), those biases will be replicated — and potentially amplified — in production.

Fairness testing isn't just ethical; it's increasingly a legal requirement. The EU AI Act, the NYC Local Law 144 (for hiring algorithms), and the CFPB's stance on fair lending all require demonstrable bias auditing.

The Three Core Metrics

1. Demographic Parity (Statistical Parity)

Definition: The probability of receiving a positive prediction should be the same regardless of group membership.

Formula (stdlib):

# For each group g: positive_rate(g) = count(ŷ=1 in g) / count(g)
# Ratio = min(rates) / max(rates)
# Pass when ratio ≥ 0.80 (the four-fifths rule)

When to use: When you care about *equal representation* in outcomes. Appropriate for hiring, lending, and resource allocation where unequal impact requires justification.

When NOT to use: When base rates legitimately differ between groups. Medical diagnosis, for example — disease prevalence varies by demographics, so forcing equal positive rates would misdiagnose patients.

Code:

from src.fairness import demographic_parity_ratio

groups = ["male", "male", "female", "female", "female"]
y_pred = [1, 1, 1, 0, 0]
ratio = demographic_parity_ratio(groups, y_pred)
print(f"DP ratio: {ratio:.2f}")
# male rate: 2/2 = 1.0,  female rate: 1/3 = 0.33
# ratio = 0.33 / 1.0 = 0.33 → FAILS four-fifths rule

2. Equalized Odds

Definition: The true positive rate (TPR) and false positive rate (FPR) should be equal across groups.

Why both TPR and FPR?

• Equal TPR means the model is equally good at catching true positives across groups. A cancer screening model that catches 95% of cases in one demographic but only 70% in another is dangerous.
• Equal FPR means the model is equally likely to generate false alarms across groups. A fraud detection model that falsely flags 20% of transactions from one group but only 2% from another creates a discriminatory customer experience.

Formula (stdlib):

# For each group g:
#   TPR(g) = TP(g) / (TP(g) + FN(g))
#   FPR(g) = FP(g) / (FP(g) + TN(g))
# Max TPR gap = max(TPR) - min(TPR)
# Max FPR gap = max(FPR) - min(FPR)
# Pass when both gaps ≤ 0.10

Code:

from src.fairness import equalized_odds_gap

groups = ["A", "A", "A", "A", "B", "B", "B", "B"]
y_true = [1, 1, 0, 0, 1, 1, 0, 0]
y_pred = [1, 1, 0, 0, 1, 0, 0, 0]  # Group B misses one positive
tpr_gap, fpr_gap = equalized_odds_gap(groups, y_true, y_pred)
print(f"TPR gap: {tpr_gap:.2f}, FPR gap: {fpr_gap:.2f}")
# Group A TPR: 1.0, Group B TPR: 0.5 → gap = 0.5 → FAILS

3. Disparate Impact Ratio

Definition: Identical to demographic parity ratio. Named differently because the legal concept ("disparate impact") has specific implications in US employment and lending law.

The four-fifths rule: if the selection rate for any protected group is less than 80% of the rate for the most-selected group, there is evidence of adverse impact.

The Full Audit Workflow

from src.fairness import audit_fairness

report = audit_fairness(
    groups=test_df["demographic_group"].tolist(),
    y_true=test_df["label"].tolist(),
    y_pred=predictions.tolist(),
    dp_threshold=0.80,
    eo_tpr_threshold=0.10,
    eo_fpr_threshold=0.10,
)

print(report.summary())
# Output:
# Fairness audit: PASSED
#   [PASS] demographic_parity_ratio: 0.8523 (threshold: 0.80)
#   [PASS] equalized_odds_tpr_gap: 0.0412 (threshold: 0.10)
#   [PASS] equalized_odds_fpr_gap: 0.0187 (threshold: 0.10)
#   [PASS] disparate_impact_ratio: 0.8523 (threshold: 0.80)

Subgroup Deep Dive

When the audit *fails*, you need to understand *which* group is affected and *why*:

from src.fairness import compute_subgroup_metrics

metrics = compute_subgroup_metrics(groups, y_true, y_pred)
for group, scores in metrics.items():
    print(f"{group}: acc={scores['accuracy']:.3f}, "
          f"prec={scores['precision']:.3f}, rec={scores['recall']:.3f}, "
          f"f1={scores['f1']:.3f}, n={scores['count']:.0f}")

This table often reveals the root cause:

• Low recall in one group → the model under-predicts positives for that group (possibly due to under-representation in training data).
• High FPR in one group → the model over-predicts positives for that group (possibly due to label noise or proxy features).

Choosing Protected Attributes

Test on all attributes that could create legal or ethical risk:

Proxy features: Be aware that even if you don't include protected attributes directly, the model can learn to approximate them from correlated features (zip code → race, name → gender). Test fairness on the *actual protected attribute*, not just the features in the model.

Intersectionality

Groups can be intersectional. A model might be fair for "women" and "Black applicants" individually but biased against "Black women" specifically. Test intersections:

# Create intersectional groups
groups = [f"{row['gender']}_{row['race']}" for _, row in df.iterrows()]
report = audit_fairness(groups, y_true, y_pred)

The trade-off: more intersections = smaller group sizes = noisier estimates. You need at least 50–100 samples per intersectional group for reliable results.

What to Do When Fairness Checks Fail

Decision framework:

1. Is the disparity due to training data imbalance?
   → Collect more data for underrepresented groups
   → Apply class-reweighting or oversampling

2. Is the disparity due to proxy features?
   → Identify and remove or decorrelate proxies
   → Use adversarial debiasing techniques

3. Is the disparity due to legitimate base-rate differences?
   → Document the justification
   → Switch from demographic parity to equalized odds
   → Consider calibration-based fairness

4. Is the disparity small but persistent?
   → Apply post-processing threshold adjustment per group
   → Trade off a small amount of overall accuracy for fairness

5. None of the above work?
   → The model should not be deployed for this use case

Integration with CI/CD

Add fairness as a required gate in your deployment pipeline:

# In your CI test file
def test_fairness_audit():
    """Block deployment if fairness checks fail."""
    report = audit_fairness(
        groups=test_groups,
        y_true=test_labels,
        y_pred=model_predictions,
    )
    assert report.passed, (
        f"Fairness audit failed:\n{report.summary()}\n\n"
        f"Failures: {[f.name for f in report.failures]}\n"
        f"See guides/fairness-testing.md for remediation steps."
    )

Set the gate as non-blocking initially while you calibrate thresholds, then make it blocking once you've established baselines. Failing a fairness gate should be treated with the same severity as a security vulnerability.