labTwo gates. A check before the work and a review before the release
The lab's work cycle has nine steps, and two of them are gates. Without the legal and ethical check the work does not start; without an independent review the result is not released.
Every piece of the lab's work goes through one analytical cycle of nine steps. Seven steps make up the work itself. Two steps stop it if something is wrong: these are the gates. The first stands before the work starts, the second before the result is released. On the cover they are marked in red: on the site that is the colour of a boundary that work does not cross without a check.
The first gate: the legal and ethical check
The request that starts a piece of work is checked first. Without clearance the work does not start, and the request is declined.
The check answers five questions:
- is the aim of the work lawful
- does it concern people only in their public role: owner, director, signatory, party to an act
- can what is needed be collected from sources anyone may lawfully read
- does anyone who will do the work have a conflict of interest
- does the client agree to be named if it asks for a recommendation
A refusal at this step costs the least: no data has been collected yet, and nothing has to be stopped halfway.
The second gate: the quality check and review
The draft of the result, with its conclusions, scenarios and indicators, goes to a reviewer from outside the team that prepared it. Without an independent review there is no release, and a remark from the reviewer sends the work back.
The reviewer checks:
- whether the conclusions follow from the sources and the reasoning has no leaps
- whether no fewer than two explanations were weighed against the data and the alternatives in the note are real
- whether the confidence of a conclusion is no higher than the grades of its sources allow
A conclusion about a specific organisation or a specific person also passes a legal review. Strategic assessments, the annual forecast and public conclusions about specific organisations go through an alternative analysis group (red team) as well. It looks for other explanations, hidden assumptions and signs of groupthink.
Why the checks stand at the start and at the end
Work that must not be done is better not started, and the first gate stops it before any data is collected. A result that does not rest on its sources must not be released, and the second gate stops it before it reaches the reader.
Between the gates the work goes on without separate permissions: the analyst does not wait for approval at every step.
What a gate does not guarantee
The reviewer checks the reasoning and its link to the sources. The reviewer does not repeat the collection of data, and a review can miss an error in a source itself. That is why every source carries a grade, and why a debrief follows the event: what was right, what was missed and which assumptions proved wrong.
The lab's public material is released after an independent review and an editorial check. This is one of the rules of public communication in the lab's principles.