Blog

# Biases in estimating probabilities

Understanding how often security breaches happen is important to understanding how many resources to allocate to preventing them. This can be tricky because there's not much reliable data about how often security breaches happen. People also don't estimate probabilities very well, so in the absence of good data we're likely to make mistakes that can lead to either too much or too little being spent. This problem isn't limited to just information security, of course. It also complicates things any time we don't have good estimates of probabilities.

I recently came across an interesting discussion of this in a book by the CIA: Psychology of Intelligence Analysis. Here's the book's summary of its Chapter 12, "Biases in Estimating Probabilities," and these comments seem to apply to information security just as well as it applies to intelligence analysis:

In making rough probability judgments, people commonly depend upon one of several simplified rules of thumb that greatly ease the burden of decision. Using the "availability" rule, people judge the probability of an event by the ease with which they can imagine relevant instances of similar events or the number of such events that they can easily remember. With the "anchoring" strategy, people pick some natural starting point for a first approximation and then adjust this figure based on the results of additional information or analysis. Typically, they do not adjust the initial judgment enough.

Expressions of probability, such as possible and probable, are a common source of ambiguity that make it easier for a reader to interpret a report as consistent with the reader's own preconceptions. The probability of a scenario is often miscalculated. Data on "prior probabilities" are commonly ignored unless they illuminate causal relationships.

So if you're interested in how people mis-estimate probabilities and ways to deal with this, this CIA book actually seems to have a fairly good discussion of it. And the price (free) is certainly right.