How to version models, promote through stages, and roll back safely.
After 50 training runs you have dozens of model files scattered across directories. When someone asks "which model is serving in production?" you grep through logs and hope for the best.
A model registry solves this by:
┌──────┐ ┌─────────┐ ┌────────────┐ ┌──────────┐
│ NONE │ ──▶ │ STAGING │ ──▶ │ PRODUCTION │ ──▶ │ ARCHIVED │
└──────┘ └─────────┘ └────────────┘ └──────────┘
▲ │
└────────────────────────────────────┘
(rollback path)Rule: Only ONE version can be PRODUCTION at a time per model name. Promoting a new version auto-archives the current one.
from src.model_registry import ModelRegistry
registry = ModelRegistry(base_dir="./model_registry")
# Create the model container (once per model name)
registry.create_model("churn-predictor", description="Customer churn binary classifier")
# Register a version from a training run
v1 = registry.register_version(
model_name="churn-predictor",
run_id="abc123def456",
metrics={"accuracy": 0.87, "f1": 0.82, "auc": 0.91},
params={"model": "logistic_regression", "C": 1.0},
artifact_path="./models/churn_lr_v1.pkl",
description="Baseline: L2-regularized logistic regression",
)from src.model_registry import ModelStage
registry.transition_stage(
"churn-predictor", v1.version, ModelStage.STAGING,
user="ml-engineer",
reason="Passed offline evaluation, sending to shadow traffic test",
)registry.transition_stage(
"churn-predictor", v1.version, ModelStage.PRODUCTION,
user="ml-lead",
reason="Shadow traffic showed no regression vs current baseline",
)v2 = registry.register_version(
model_name="churn-predictor",
run_id="789ghi012jkl",
metrics={"accuracy": 0.92, "f1": 0.89, "auc": 0.95},
params={"model": "xgboost", "max_depth": 6, "lr": 0.01},
artifact_path="./models/churn_xgb_v2.pkl",
description="XGBoost with tuned hyperparameters, +5% accuracy",
)
# Stage it
registry.transition_stage(
"churn-predictor", v2.version, ModelStage.STAGING,
user="ml-engineer",
reason="Outperformed v1 in offline eval by 5% accuracy",
)
# Promote to production — v1 is automatically archived
registry.transition_stage(
"churn-predictor", v2.version, ModelStage.PRODUCTION,
user="ml-lead",
reason="A/B test: +4% conversion lift over v1",
)# v2 is degrading in production — roll back to v1
rolled_back = registry.rollback(
"churn-predictor", to_version=v1.version,
user="on-call-engineer",
reason="v2 showing 15% increase in false positives since Tuesday",
)The rollback method handles the stage transitions automatically:
1. Archives the current production version (v2)
2. Moves the target version (v1) from ARCHIVED → STAGING → PRODUCTION
3. Records audit entries for every stage change
prod = registry.get_production_version("churn-predictor")
if prod:
print(f"Production: v{prod.version}, accuracy={prod.metrics['accuracy']}")
print(f"Artifact: {prod.artifact_path}")for v in registry.list_versions("churn-predictor"):
print(f" v{v.version} [{v.stage}] — {v.description}")staging = registry.list_versions("churn-predictor", stage=ModelStage.STAGING)
archived = registry.list_versions("churn-predictor", stage=ModelStage.ARCHIVED)report = registry.version_history_markdown("churn-predictor")
print(report)Use these as guidelines for when to promote:
| Gate | NONE → STAGING | STAGING → PRODUCTION |
|---|---|---|
| Offline metrics | Beat current prod on key metric | N/A (already passed) |
| Data validation | Training data passes quality checks | N/A |
| Shadow traffic | N/A | No regression vs current prod |
| A/B test | N/A | Statistically significant improvement |
| Latency check | N/A | p99 latency within SLA |
| Bias audit | N/A | No disparate impact on protected groups |
| Approval | ML engineer | ML lead or product owner |
Combine the registry with the experiment logger for a complete workflow:
from src.experiment_logger import ExperimentLogger
from src.model_registry import ModelRegistry, ModelStage
exp = ExperimentLogger(backend="stdlib")
registry = ModelRegistry()
# Train and track
with exp.run("churn-model", run_name="xgb-v3") as run_id:
exp.log_params({"model": "xgboost", "max_depth": 8})
# ... training code ...
exp.log_metric("accuracy", 0.94)
exp.log_metric("f1", 0.91)
# Register the model version using the run_id
v3 = registry.register_version(
"churn-predictor",
run_id=run_id,
metrics={"accuracy": 0.94, "f1": 0.91},
params={"model": "xgboost", "max_depth": 8},
description="XGBoost v3, deeper trees",
)
# Now promote through stages as described aboveThe local registry stores everything as JSON:
model_registry/
├── models/
│ ├── churn-predictor.json # Model metadata
│ └── fraud-detector.json
└── versions/
├── churn-predictor/
│ ├── v1.json # Version metadata + audit trail
│ ├── v2.json
│ └── v3.json
└── fraud-detector/
└── v1.jsonFor production, migrate to MLflow Model Registry or a managed service (SageMaker, Vertex AI) — the workflow logic stays the same.
Get the full Experiment Tracking Setup and unlock everything.
Get the complete guide with every chapter unlocked, including code samples, diagrams, and best practices.
Access all interactive tools with complete data, all workload profiles, and the full scenario library.
Downloadable source code, configuration files, and working examples from every chapter.
Free updates for life. Every new chapter, tool, and improvement included.