labLogical errors in open-source investigation
An error in an investigation arises more often in the step from data to conclusion than in the data itself. Eight such errors, with examples and with what the method holds against each.
Open-source data is rarely false: a date in a registry and the content of a page in an archive are usually right. An error more often arises in the step from data to conclusion, and it is the analyst who makes it. Below are eight errors that occur more often than others, and what the method holds against each.
1. Confirming the hypothesis
The error. The hypothesis appeared early, and from then on the search is for what confirms it. What contradicts it goes unnoticed or gets explained away.
How it looks. A dossier with not one fact against the conclusion.
What the method does. Gaps are a mandatory section. A verification task is phrased as a question to which "no" is a possible answer. The second analyst confirms a record, not a conclusion.
2. A relation where there is none
The error. Two objects landed on one diagram, and a line is drawn between them. A shared surname, an address in one office building, the same bank.
How it looks. A diagram of a hundred nodes in which everything is connected to everything.
What the method does. A relation without a basis is not drawn. A presumption is called a presumption and is drawn differently. Two or more unconfirmed relations in a row are marked as a finding that rests on nothing.
3. Rarity of the event
The error. A coincidence is taken for a relation without asking how common it is. Two people with one surname in one city are not rare if the surname is common.
How it looks. "The same Smith."
What the method does. A record of a match carries the basis for treating both mentions as one object. Until there is a basis, there are two records.
4. After, therefore because of
The error. Event B happened after event A, therefore A caused it.
How it looks. A chronology read as a causal chain.
What the method does. The order of events on the timeline is drawn only where the relation was established. Adjacent milestones without an established relation get no arrow.
5. One source under five names
The error. Five publications repeat one statement. It is taken to have five sources.
How it looks. "Confirmed by several sources", where all of them cite one.
What the method does. A source is a record. Five links to one document give one source record, and that is visible on the graph.
6. Silence of the source
The error. The registry has no entry, therefore the fact did not occur.
How it looks. "The company has no branches" — on the strength of one registry.
What the method does. The absence of an entry is recorded as the absence of an entry, with a date and a source. It does not count as a fact about the object, and in the report it is a gap.
7. False precision
The error. An estimate is given a number: "reliability 80". The number looks like a measurement, although it measures nothing.
How it looks. Percentages in a report with no formula.
What the method does. A figure is either computed from the data and explainable, or absent. Verification is four marks, each of which is checked by looking at the record.
8. The anchor of the first finding
The error. The first thing found sets the frame, and everything afterwards is fitted to it.
How it looks. A dossier whose structure repeats the order of the search instead of the structure of the case.
What the method does. Records exist independently of the order in which they were found. The layout of the graph is computed from the relations, and the history of the work does not affect it.
In general
The analyst's attention does not remove any of the eight errors: over a long investigation attention weakens. What protects is an arrangement of the work in which a wrong step cannot be recorded so that it looks verified. That is why the method keeps the state of verification on every record and computes it from the data.