Experiment Tracking Pack
Experiment tracking with W&B and MLflow, custom dashboards, metrics comparison, and reproducibility tooling.
Product Preview
Try the interactive reader and demo tools below, or get the full product with all content unlocked.
Interactive Reader (Free Preview) ⚙ Try Demo Tools Download Free SampleFile Structure 18 files
Documentation Preview README excerpt
Experiment Tracking Pack
One experiment-tracking interface, three interchangeable backends. Instrument
your ML training loop once and send the results to Weights & Biases,
MLflow, or a zero-dependency local SQLite store — switching between them
with a single line of configuration.
The W&B and MLflow SDKs are imported lazily, so import exp_tracking (and the
entire test suite) works with nothing but the Python standard library
installed. You only need wandb or mlflow when you actually use that backend.
Features
- Unified
ExperimentTrackerinterface —start_run,log_params,
log_metrics, log_artifact, end_run, plus list_runs / get_metric_history
for reading results back.
- Three backends, identical API:
CustomTracker— local, SQLite-backed, pure stdlib (great for tests/CI/offline).WandbTracker— Weights & Biases, with artifact and sweep support.MlflowTracker— MLflow Tracking, localmlruns/or a self-hosted server.run()context manager that always finalizes a run —FINISHEDon success,
FAILED (and re-raised) on exception.
- Hyperparameter sweeps — grid and random search expansion, plus
build_wandb_sweep_config for W&B Bayesian sweeps.
- Cross-backend comparison —
compare_runsbuilds a table from any backend,
rendered as ASCII, Markdown, or CSV, with best_run to pick the winner.
- Config-driven backend selection — choose the backend from a YAML/JSON file.
- Metric validation — non-numeric / NaN / inf values are rejected with a
clear local error before they reach a backend.
Installation
The pack itself needs no installation — drop src/exp_tracking/ onto your
PYTHONPATH (the examples and tests do this automatically). Install a backend
SDK only if you use it:
pip install wandb # for the W&B backend (then: wandb login)
pip install mlflow # for the MLflow backend
pip install pyyaml # only if you load a .yaml config (JSON needs nothing)
# the local SQLite backend, sweeps, and comparison need no third-party packages
Quick Start
from exp_tracking import get_tracker, compare_runs, format_table
# Pick a backend by name. "custom" runs locally with zero setup.
tracker = get_tracker("custom", db_path="runs.db", project="image-classification")
# Track a training loop. The context manager finalizes the run for you.
with tracker.run(run_name="adam-lr1e-3", config={"lr": 1e-3, "epochs": 10}) as run:
for epoch in range(10):
loss = 1.0 / (epoch + 1)
tracker.log_metrics({"train_loss": loss, "val_accuracy": 0.5 + epoch / 20},
step=epoch)
# Compare every run in the project and print a table.
table = compare_runs(tracker, metric_keys=["val_accuracy", "train_loss"])
*... continues with setup instructions, usage examples, and more.*