labWords of probability. Why the lab's conclusions carry no percentages
Probability in the lab's notes is written as a word from a fixed scale. The scale has seven words, each with its own range, and confidence in the grounds is stated separately.
The lab writes the probability of an event in one of seven words: "almost no chance", "unlikely", "possible", "roughly even", "likely", "very likely", "almost certain". There are no other words for probability in its conclusions: the lab's methodology fixes the scale. The scale attaches no numbers to the words.
Where the scale comes from
In 1951 American intelligence issued an estimate on a possible invasion of Yugoslavia. It used the words "serious possibility". Sherman Kent, then on the CIA's Board of National Estimates, later asked his colleagues what odds they had in mind. The answers ranged from one chance in five to four in five.
Kent described the case in his essay "Words of Estimative Probability" (1964) and proposed tying the words to ranges. That is how the scale in the analytic standards of the US intelligence community (directive ICD 203) and the probability scale of British intelligence are built today. Both have seven steps, and the lab's scale has the same structure.
Why a word and not a number
A number promises a precision the judgement does not have. "A probability of 70" looks like a measurement and measures nothing: it is the same false precision as a reliability percentage in a report with no formula behind it.
A number also invites arithmetic: adding, averaging and multiplying the estimates of different analysts. Arithmetic on judgements that have no data behind them gives a result that looks stricter than the judgements it started from.
A word, in turn, has to mean the same thing for every analyst. So each word in the lab's methodology is tied to its own range, and the ranges do not overlap. The text carries the word.
Why there is no "impossible" and no "certain"
The scale starts at "almost no chance" and ends at "almost certain". "Impossible" and "certain" would state a fact, and a fact is written differently: in the Fact block of a note, with its sources and their grades.
If an analyst wants to write "certain", it is a reason to check whether an established fact is in front of them. An established fact needs no estimate of probability.
Probability and confidence
Probability is about the event. Confidence is about the quality of the grounds for the estimate. In a note they stand side by side but apart: "likely, confidence low".
Such an entry says more than a single word. The reader sees that there are grounds for "likely" but that they are weak: a single source, gaps in the data or substantial assumptions. Confidence is high, medium or low, and each level is described in the methodology.
How the scale is checked
The words of the scale are checked against the forecast register. If noticeably fewer forecasts marked "likely" come true than its range promises, the analysts overrate their grounds. This check is called calibration.
Common mistakes
- A word outside the scale. "Cannot be excluded", "there are grounds to believe", "most probably": the reader does not know where to place them, and every reader places them differently.
- Stacking. "Possibly likely" or "very likely, though not certain". One word is enough.
- The desired as the likely. A desired scenario is never presented as the most likely one. This is one of the lab's principles, and the reviewer checks it separately.