Research question

Can a small, independent sample reveal whether “complete” statuses are supported by the required evidence? The objective is not to turn a few reviewed files into a precise estimate of every error. It is to test whether the closure rule is being applied consistently and decide whether broader inspection or process repair is warranted.

The Census Bureau’s nonresponse follow-up quality-control work used reinterviews, matching, and clerical review to compare production results. NIST’s acceptance-sampling handbook explains that a sample can inform a disposition decision but is not the same as estimating the overall quality of a population. Those principles support independent comparison and explicit decision rules, while warning against claims that a small sample proves the whole queue is correct.

Define closure and the review population

Before sampling, define a closed item as a specific expected result, evidence location, acceptance authority, and closure timestamp. A status change alone is not evidence. Freeze the population of items marked complete in a bounded period, including reopened items and items closed by automation.

Choose the sample using a documented method that the process owner cannot manipulate after seeing the records. Simple random selection may be adequate for a homogeneous queue. A risk-stratified sample can deliberately include high-consequence types, automated closures, and new workflows, but its results should not be reported as though every item had the same selection probability.

The reviewer records one of four outcomes: supported, unsupported, indeterminate because evidence is unavailable, or not applicable because the sampled item was outside the stated rule. Preserve both the original status and review outcome. “Indeterminate” should not be silently converted to supported or unsupported.

Measure reviewer agreement

Start with a small calibration set reviewed independently by two people. Compare their outcomes before discussing them. Disagreement may show that the rule, evidence expectations, or reviewer guidance is ambiguous. Reconcile the cases with the accountable owner, document the clarified rule, and only then review the main sample.

For the main sample, report population size, sample size, selection method, reviewed count, each outcome count, exclusions, and reviewer disagreements. If the study uses a formal acceptance rule, specify it in advance and obtain statistical review appropriate to the consequence. Do not invent a “95% accurate” headline from the percentage supported in a small convenience sample.

When an unsupported item appears, record the defect class: missing output, inaccessible evidence, wrong approver, premature closure, stale evidence, or reopened outcome. Route correction separately from the research record. Changing the production item should not erase what the sample observed.

Escalation logic and limits

One serious defect may justify immediate owner review even when it does not support a population-wide rate. Repeated defects of the same class can justify broader inspection or a control change. Conversely, a sample with no defects does not prove zero defects; it provides bounded evidence under the stated design.

An operations coordinator can maintain the population snapshot, execute the approved selection method, assemble evidence, and schedule independent review. The coordinator should not review their own closures as the only reviewer, lower the evidence rule after selection, or accept legal, security, financial, or compliance outcomes.

Protect sensitive evidence by recording controlled links and minimal classifications rather than copying source files into the study. Review access should match the sampled records, and the final report should aggregate results unless row-level disclosure is necessary.

Conclusion

Completion sampling is useful when it tests a frozen closure rule against independent source evidence and reports uncertainty honestly. Its best output is a decision about what to inspect or repair next, rather than a decorative quality score detached from the sampling design.

OperationsCoordinator can coordinate the population snapshot, evidence packet, and maker-distinct review through our operations support services. Acceptance authority remains with the named process owner.

Sources and limits

These sources do not provide a universal sample size or acceptance threshold for administrative work. This article is operational research guidance, not audit, statistical, legal, compliance, or personnel advice.