Intermittent operations demand
Intermittent operations demand should be evaluated as a sequence of zero and nonzero periods, not as a smooth average workload. A useful forecast test preserves the timing of arrivals, the size of each nonzero burst, the forecast horizon, and the operational cost of being wrong.
The problem appears in low-frequency complaints, urgent approvals, warranty claims, incident escalations, and specialized service requests. A monthly average can be numerically correct while leaving the team unprepared for the weeks when work actually arrives.
The online text Forecasting: Principles and Practice describes count data and Croston's method for series with many zero values. It treats nonzero demand size and the interval between nonzero demands as separate quantities, which is the core distinction this study preserves.
Define the decision before the error measure
State what the forecast controls. The decision may be staffing a review window, reserving specialist capacity, setting a reorder point, or choosing when to ask for on-call coverage. Error should be reported in the units that decision owner uses.
Freeze the time bucket and horizon. Daily, weekly, and monthly versions of the same event stream answer different questions. A forecast made one week ahead must be compared with the value known after that week, not with a later restatement that includes delayed records.
The NIST exploratory data analysis handbook recommends examining distribution, outliers, structure, and assumptions before relying on a compact statistic. For intermittent work, that means showing the proportion of zero periods, distribution of nonzero sizes, gaps between arrivals, and known changes in intake.
Build a forecast ledger
Keep one row for every forecast origin and horizon. Record the forecast creation time, periods covered, method and version, input-data cutoff, point forecast, interval if available, actual value as first observed, revised actual value, and reason for any revision.
This ledger prevents hindsight from entering the evaluation. The accuracy chapter of Forecasting: Principles and Practice states that forecast accuracy must be tested on observations not used to fit the model. In-sample residuals answer a different question.
Use a naive baseline that the proposed method must beat. Depending on cadence, the baseline might repeat the last observed value, the value from the same period in the previous cycle, or the historical mean available at the forecast origin. Document the choice before viewing the test result.
The GAO Cost Estimating Guide calls for a documented technical baseline, assumptions, data, sensitivity analysis, risk analysis, updates with actual costs, and validation. Although its scope is program cost, those controls are directly useful for a small forecast ledger.
Choose errors that survive zero demand
Mean absolute error reports the average size of the miss in work units. Root mean squared error gives larger misses more influence. Neither divides by the actual value, so both remain defined when actual demand is zero.
Percentage error is fragile here. Dividing by zero is undefined, and dividing by a small nonzero count can create an extreme percentage that dominates the summary. The NIST scale measures explains why spread and robust summaries should be chosen with the data distribution in mind.
Also report forecast bias as the signed total or mean error. Positive and negative misses can cancel in an average absolute summary, but persistent overforecasting and underforecasting have different capacity consequences.
Show service consequences beside statistical errors. Count periods when actual demand exceeded reserved capacity, unused reserved units, cases deferred, and emergency coverage activations. These are descriptive outcomes, not proof that the forecast caused service performance.
Account for calendar and structural changes
Intermittent demand may still contain seasonality, scheduled deadlines, or reporting cycles. The Census seasonal adjustment guidance describes seasonal adjustment as estimating and removing recurring seasonal effects so other movement is easier to study.
The BEA seasonal adjustment page documents methods and resources used for official economic estimates. A small operations team does not need that machinery, but it should separate known calendar effects from unexplained bursts when enough history exists.
Record structural breaks such as a new intake channel, pricing change, client migration, policy revision, or system outage. The NIST change-point discussion shows why a shift in process level should not be hidden inside a single long-run average.
Interpretation and operating boundaries
A lower historical error does not guarantee better future staffing. The result applies to the tested horizons, decision rule, data vintage, and demand regime. Report uncertainty and the failure conditions that matter to the owner.
The Federal Reserve uncertainty guidance records how uncertainty and risks around projections are considered separately from point estimates. The setting differs, but the distinction is sound for operations: a point forecast is not a promise.
Use the Research index for related measurement work. The preceding study on event-log quality addresses whether recorded process history is trustworthy, while the capacity planning guide covers the service workflow around capacity records.
A repeatable comparison
Choose a fixed training window, test window, horizons, baseline, candidate methods, and error measures. Run every method against the same forecast origins. Publish all origins, including zero periods and major misses.
Segment only on characteristics known at the forecast time. If request type is corrected after completion, preserve the original class and report that later knowledge was unavailable to the forecast.
The NIST uncertainty guidance distinguishes measurement results from the uncertainty attached to them. Forecasts are not measurements, but the reporting discipline transfers: a number without its uncertainty and assumptions is incomplete evidence.
Sources
- Forecasting count data
- Evaluating forecast accuracy
- Forecasting data and methods
- NIST exploratory data analysis
- NIST measures of scale
- GAO Cost Estimating Guide
- Census seasonal adjustment
- BEA seasonal adjustment
- NIST change-point discussion
- Federal Reserve forecast uncertainty
- NIST Technical Note 1297
These sources support forecasting methods and reporting controls. They do not establish a universal staffing formula or prove that one method will fit every intermittent series.