← Back to all products

Forecast Accuracy Monitoring Framework

$990

Universal time-series forecast monitoring: one TVF + dashboards for MAE/RMSE/MAPE, bias, horizon buckets, and fair multi-system comparison. Deploy in 30 minutes for any forecast.

17 filesv1.0.0
PythonMarkdownSQLYAMLJSONDatabricks

Product Preview

Try the interactive reader and demo tools below, or get the full product with all content unlocked.

⚙ Try Demo Tools

File Structure 17 files

forecast-accuracy-monitoring/ ├── LICENSE ├── README.md ├── conftest.py ├── databricks.yml ├── guide/ │ ├── 01_what-you-get.md │ ├── 02_getting-started.md │ ├── 03_architecture.md │ └── 04_support.md ├── guides/ │ └── forecast-monitoring-methodology.md ├── lib/ │ └── forecast_metrics.py ├── resources/ │ └── jobs.yml ├── src/ │ ├── dashboard/ │ │ ├── 01_headline_kpis.sql │ │ ├── 02_weekly_mae_trend.sql │ │ └── 03_error_by_horizon.sql │ └── tvf/ │ └── dashboard_forecasts.sql └── tests/ └── test_forecast_metrics.py

Documentation Preview README excerpt

Forecast Accuracy Monitoring Accelerator

Universal time-series forecast monitoring. Plug in any forecast + actuals

and get MAE / RMSE / MAPE dashboards with horizon-bucketed error, weekly trends,

and fair multi-system comparison — deploy in about 30 minutes for heating,

wind, solar, price, load, or any other time series.

The idea

All dashboard queries call one table-valued function (dashboard_forecasts)

that unions your forecast systems with actuals and computes error + horizon.

Zero SQL duplication: to add a forecast system you add one UNION ALL block.

Why "fair comparison" matters

If system A forecasts 48h ahead and system B only 6h ahead, comparing their raw

MAE is misleading — long-horizon errors are larger. The framework caps every

system at the minimum of each system's max horizon before computing KPIs, so

comparisons are apples-to-apples. The same logic is in lib/forecast_metrics.py

and unit-tested.

What's inside

  • Databricks Asset Bundle — daily job that deploys the TVF and refreshes KPI

datasets.

  • One TVF + three dashboard queries — headline KPIs, weekly MAE trend, and

error-by-horizon, all parameterized by :catalog, :schema, :lookback_days.

  • A dependency-free metrics library (lib/forecast_metrics.py) — MAE, RMSE,

MAPE, bias, bias share, horizon bucketing, and the fair-max-horizon rule.

  • An offline test suite (tests/, 7 passing tests).

Quickstart


pip install pytest && pytest tests/ -v      # validate metrics offline
databricks bundle deploy -t dev -p <profile>

Provide forecast_primary, forecast_secondary, actuals, and (optional)

weather_observations tables in your :schema — schemas documented in

guide/03_architecture.md.

License

MIT — see LICENSE.

Code Sample .py preview

conftest.py import os,sys sys.path.insert(0, os.path.dirname(__file__))