Recurring-work estimate calibration
An estimate becomes useful for learning only when its original value survives contact with the outcome. Replacing the forecast after work starts produces a tidy record but removes the error needed for calibration. The GAO Cost Estimating Guide calls for documented assumptions, data, sensitivity, risk, and updates using actual costs, offering a transferable discipline without prescribing an effort model for small teams.
Use one completed or censored work item as the unit. Record the frozen estimate, estimate date, work class, scope version, estimator role, units, range or point value, actual outcome, evidence coverage, and reasons for authorized scope change. Do not mix elapsed time, active effort, and cost in one error calculation.
Preserve comparable classes
Define recurring work narrowly enough that items share an acceptance rule and measurement unit. A routine data update and an exception investigation may use the same queue but require different models. The NASA Cost Estimating Handbook emphasizes scope, methods, documentation, uncertainty, and independent assessment.
Freeze a cohort by estimate date and retain incomplete items at the cutoff. For point estimates, calculate signed error to show direction and absolute error to show magnitude. For ranges, report whether the outcome fell below, inside, or above the range, while keeping the width visible.
Segment results by stable work class and estimate version. Do not create many tiny segments after seeing outcomes. A small cohort can reveal a repeatable directional bias, but it cannot establish a universal adjustment factor.
Revise the model, not history
Review large misses against evidence: changed scope, missing intake, dependency delay, wrong work class, access constraint, or weak original assumptions. Preserve the old model and document the effective date of any new one. The GAO financial-system review highlights actual-versus-planned variance review and lessons for future estimates.
Report estimate coverage, outcome coverage, signed error, absolute error, range containment, scope-change rate, and indeterminate cases. A model can appear accurate if hard items lack outcomes, so show missingness beside every summary. The NIST information quality standards support clear context and reproducible analysis.
An operations coordinator can freeze estimates, reconcile outcomes, and prepare calibration views. The accountable owner chooses the planning model, contingency, commitments, and any staffing response. Our operations support services can maintain the evidence, and the Research index contains complementary methods.
Sources and limits
- GAO Cost Estimating Guide, for documented baselines, assumptions, actuals, and updates.
- NASA Cost Estimating Handbook, for scope, methodology, uncertainty, and assessment.
- NASA planning guidance, for current estimating and planning resources.
- GAO financial-system review, for planned-versus-actual variance and lessons.
- GAO VA modernization review, for comprehensive, documented, accurate, and credible estimates.
- GAO DOE estimating review, for the steps behind high-quality estimates.
- NIST process characterization, for defined process inputs and outputs.
- NIST Research Data Framework, for provenance and documented change.
- NIST information quality standards, for clear, reproducible analytic results.
- GAO evidence guide, for sufficient and appropriate evidence.
These sources support calibration discipline, not a universal effort estimate or promised accuracy. This article is operational research guidance, not financial, project-management, staffing, or contractual advice.