On method: how to check a claim

Every volume of Principles of Practice ends by naming the statistics its field repeats most often, and explaining why they are not used in the book. This is that lesson, and it is here rather than in an appendix because it is the most transferable thing in the course.

Operations writing is unusually full of numbers with no source. The eight second attention span, widely cited, traces to no study. "If you cannot measure it, you cannot manage it" is attributed to Drucker, who neither wrote it nor believed it; his actual position was closer to the opposite. Various acquisition-versus-retention cost ratios circulate with no methodology attached to any of them. Several widely quoted automation failure rates resolve, on inspection, to a consultancy survey with an undisclosed sample.

The check is four questions and it takes about ten minutes.

Who produced this, and what were they selling? What was the sample, and of what population? What exactly was measured, in units, and does that match the claim being made from it? And has anyone independent reproduced it?

A claim that survives all four is usable. A claim that fails one is not necessarily false, but it is not evidence, and the difference matters when you are about to reorganise a department around it.

NBOP applies this to itself and publishes the results. The standard states where its own evidence is thin, the job task analysis will not publish domain weights below a floor that was fixed before collection began, and the count of responses excluded from that analysis is published alongside the weights rather than quietly omitted. A body that asks operators to audit their instruments and does not audit its own is not a standards body. It is a publisher with a logo.