Forecast accuracy
Six indicators that evaluate how close the forecast was to actual demand. They are only computed over closed versions and elapsed periods; future periods and open versions are excluded.
Error convention
error = D − F
where D is actual demand and F is the forecast. KPIs are computed per SKU and then averaged (each SKU carries equal weight). "Cumulative" averages across several closed versions.
- Positive error (D > F): we fell short → risk of stockout.
- Negative error (D < F): we over-forecasted → risk of surplus.
Metrics catalogue
| KPI | What it measures | Unit | Good when | Formula |
|---|---|---|---|---|
| BIAS | Systematic bias (under vs. over-forecasting) | Units | ≈ 0 | (1/n) · Σ(Dᵢ − Fᵢ) |
| MAE | Mean absolute error magnitude | Units | Minimum | `(1/n) · Σ |
| MAPE | Mean absolute percentage error | % | < 20 % | `(1/n) · Σ( |
| ECM (= MSE) | Mean squared error; penalises large errors | Units² | Minimum | (1/n) · Σ(Dᵢ − Fᵢ)² |
| RMSE | Square root of ECM; same unit as demand | Units | Minimum | √(ECM) |
| Accuracy | Mean per-SKU accuracy with floor at 0 | % (0–100) | Maximum | `(1/n) · Σ max(0, 100 − ( |
AInventory reports ECM (Error Cuadrático Medio) and MSE (Mean Squared Error) as the same metric — both use the formula (1/n) · Σ(Dᵢ − Fᵢ)². RMSE is its square root: RMSE = √(ECM).
Detailed descriptions
BIAS
BIAS is the arithmetic mean of the signed errors. Because positive and negative errors cancel each other out, it must always be read alongside MAE.
- BIAS = 0: the model has no systematic bias.
- BIAS > 0: persistent under-forecasting (demand consistently exceeded the forecast).
- BIAS < 0: persistent over-forecasting (the forecast consistently exceeded demand).
Example: SKU A: D=100, F=120 → error=−20. SKU B: D=80, F=70 → error=+10. BIAS = (−20+10)/2 = −5.
MAE
MAE averages the absolute values of the errors. It is the most intuitive magnitude measure because it is expressed in the same units as demand.
Example (continued): MAE = (|−20|+|10|)/2 = (20+10)/2 = 15 units.
MAPE
MAPE expresses the error as a percentage of actual demand. SKUs with D = 0 are excluded to avoid division by zero.
| Range | Rating |
|---|---|
| < 10 % | Very good |
| 10 – 20 % | Good |
| 20 – 50 % | Fair |
| > 50 % | Poor |
Example: SKU A: (20/100)×100 = 20 %. SKU B: (10/80)×100 = 12.5 %. MAPE = (20+12.5)/2 = 16.25 %.
ECM (= MSE)
ECM penalises large errors quadratically: a double error carries four times the weight. It is primarily used as an intermediate step to compute RMSE.
Example: ECM = (20²+10²)/2 = (400+100)/2 = 250 units².
RMSE
RMSE returns the error to the original units of demand. It always satisfies RMSE ≥ MAE.
- RMSE ≫ MAE: outlier errors are inflating the metric.
- RMSE ≈ MAE: errors are relatively uniform across the SKU population.
Example: RMSE = √250 ≈ 15.8 units.
Accuracy
Accuracy is the only indicator where a higher value is better. A per-SKU floor of 0 % prevents highly inaccurate SKUs from pulling the metric below zero. For this reason, Accuracy ≠ 100 − MAPE exactly.
| Range | Rating |
|---|---|
| > 90 % | Excellent |
| 80 – 90 % | Good |
| 60 – 80 % | Acceptable |
| < 60 % | Poor |
Example: SKU A: max(0, 100−20) = 80 %. SKU B: max(0, 100−12.5) = 87.5 %. Accuracy = (80+87.5)/2 = 83.75 %.
Combined reading: BIAS and MAE
| Situation | Interpretation |
|---|---|
| BIAS ≈ 0, high MAE | Large errors but no systematic bias (noise) |
| Negative BIAS, high MAE | Consistent over-forecasting (adjust downward) |
| Positive BIAS, high MAE | Consistent under-forecasting (adjust upward) |
| BIAS ≈ 0, low MAE | Good forecast quality |
Evaluate these metrics at the close of each version. In the forecast module, the cumulative view averages across all closed versions available for the selected period.