Methodology

Analytical cycle

Work starts with a client's request or a monitoring signal.

1. Request or signalThe question is set by the client or by monitoring.
   Publications

   • G.009The weak signal. From an observation to an indicator
2. GateLegal and ethical checkWithout clearance the work does not start, and the request is declined.
   Publications

   • G.010Two gates. A check before the work and a review before the release
3. Task passport and collection planThe question, the deadline, the owner of the result and the completion criterion.
   Publications

   • G.001Why an investigation template draws no relations
4. Collection and checking of dataOrigin, date, context and independent confirmation of each source. For each attached document CATK computes a SHA-256: the hash shows that the file has not changed since it was attached.
   Publications

   • G.011The source and the message. Two grades instead of one
   • G.005OPSEC. What leaves an investigator's machine
   • G.002An email address as a record. What it says and what it does not
5. Analysis and alternative hypothesesNo fewer than two explanations are weighed against the data.
   Publications

   • G.013Analysis of competing hypotheses. Contradictions decide
   • G.006Logical errors in open-source investigation
   • G.003Stylometry. An indicator, not proof
6. Conclusions and scenariosThe draft of the result, with its grounds and indicators.
   Publications

   • G.008Words of probability. Why the lab's conclusions carry no percentages
   • G.007Kinds of dossier. Three levels of the same work
   • G.004Reporting. How to document the results of an investigation
7. GateQuality check and reviewWithout an independent review there is no release. A conclusion about a specific organisation or person also passes a legal review. A reviewer's remark sends the work back for revision.
   Publications

   • G.010Two gates. A check before the work and a review before the release
8. Release and feedbackThe result goes to the reader, and the reader's questions come back.
9. Knowledge archive and calibrationOutcomes, the calibration of forecasts and the lessons are taken into account when new tasks are set.
   Publications

   • G.012The forecast register. How a forecast is checked by its outcome

Standard of an analytical note

A note answers nine questions in a fixed order. A reader who opens any of the lab's notes finds the conclusion, the facts and the alternatives in the same places.

Grading of sources

The reliability of a source and the credibility of a particular message are graded separately: a reliable source can pass on a doubtful message, and an unreliable source a true one.

The mark B2 reads: the source is usually reliable, the message is probably true. F6 means that grading is impossible, and says nothing about falsity.

Probability and confidence

Each word of the probability scale stands for its own range, and the ranges do not overlap. The extremes are not used: they would state a fact.

1. Almost no chanceThere are almost no grounds, but the scenario is not ruled out entirely.
2. UnlikelyThere are grounds, but the main scenario is a different one.
3. PossibleA real scenario if several conditions are met.
4. Roughly evenNot enough grounds to choose a side.
5. LikelyMore confirmations than refutations.
6. Very likelyA strong body of confirmation.
7. Almost certainA very high probability, though not absolute.

Confidence

• HighSeveral independent reliable sources, consistent data, a tested method.
• MediumThe data are plausible, but there are gaps, single sources or competing readings.
• LowFragmentary or doubtful data, substantial assumptions.

Probability and confidence are stated separately, for example: “likely, confidence medium”. Probability is about the event; confidence is about the quality of the grounds for judging it.

Quality check

• Significant work is reviewed by a reviewer from outside the team that prepared it.
• The reviewer checks that the conclusions match the sources and that the reasoning has no leaps of logic.
• Conclusions about specific organisations and people also pass a legal review.
• Strategic assessments, the annual forecast and public conclusions about specific organisations are checked by an alternative analysis group (red team). It looks for other explanations, hidden assumptions and signs of groupthink.
• After the event a debrief records what was right, what was missed and which assumptions proved wrong.

Forecast register

• Every forecast is recorded: the date, the wording, the probability on the scale, the horizon, the author and the date of the check.
• After the date of the check the outcome is recorded, whatever it is.
• The accumulated outcomes show calibration: how well the words of the scale match what happens.