Forecast Accuracy Monitoring Framework
Universal time-series forecast monitoring: one TVF + dashboards for MAE/RMSE/MAPE, bias, horizon buckets, and fair multi-system comparison. Deploy in 30 minutes for any forecast.
Product Preview
Try the interactive reader and demo tools below, or get the full product with all content unlocked.
⚙ Try Demo ToolsFile Structure 17 files
Documentation Preview README excerpt
Forecast Accuracy Monitoring Accelerator
Universal time-series forecast monitoring. Plug in any forecast + actuals
and get MAE / RMSE / MAPE dashboards with horizon-bucketed error, weekly trends,
and fair multi-system comparison — deploy in about 30 minutes for heating,
wind, solar, price, load, or any other time series.
The idea
All dashboard queries call one table-valued function (dashboard_forecasts)
that unions your forecast systems with actuals and computes error + horizon.
Zero SQL duplication: to add a forecast system you add one UNION ALL block.
Why "fair comparison" matters
If system A forecasts 48h ahead and system B only 6h ahead, comparing their raw
MAE is misleading — long-horizon errors are larger. The framework caps every
system at the minimum of each system's max horizon before computing KPIs, so
comparisons are apples-to-apples. The same logic is in lib/forecast_metrics.py
and unit-tested.
What's inside
- Databricks Asset Bundle — daily job that deploys the TVF and refreshes KPI
datasets.
- One TVF + three dashboard queries — headline KPIs, weekly MAE trend, and
error-by-horizon, all parameterized by :catalog, :schema, :lookback_days.
- A dependency-free metrics library (
lib/forecast_metrics.py) — MAE, RMSE,
MAPE, bias, bias share, horizon bucketing, and the fair-max-horizon rule.
- An offline test suite (
tests/, 7 passing tests).
Quickstart
pip install pytest && pytest tests/ -v # validate metrics offline
databricks bundle deploy -t dev -p <profile>
Provide forecast_primary, forecast_secondary, actuals, and (optional)
weather_observations tables in your :schema — schemas documented in
guide/03_architecture.md.
License
MIT — see LICENSE.