The dangers of a risk assessment
Performing a risk assessment is often listed as one of the first steps in information security life-cycle methodologies. Performing such a risk assessment is actually hard to do. There’s little valid data that tells how often security vulnerabilities are exploited, and it’s very hard to quantify the damage that’s done if a hacker actually exploits a vulnerability.
This means that estimating risks, which are defined to be the probability of an event multiplied by the loss associated with the event, often isn’t practical in information security. It turns out that there may actually be another reason to do such a risk assessment, even if was feasible, and this reason relates to the potential legal complications that may arise if you do a careful risk assessment. This was first noted by W. Kip Viscusi in an internal discussion paper that he wrote while at Harvard Law School that was subsequently published in the Journal of Legal Studies as "Jurors, Judges, and the Mistreatment of Risk by the Courts."
As we previously mentioned, the Hand Rule tells us that you’’e not required to spend more than the value of a risk to mitigate it. So if it will cost you $2 million to mitigate a $1 million risk and you decide not to spend the $2 million, the Hand Rule tells that you can’t be found negligent.
Viscusi’s research showed that jurors don’t properly apply negligence rules like the Hand Rule, particularly in cases where the probabilities of events are small and losses are large. Jurors seem to be offended by trade-offs between costs and risks. In Viscusi’s research, the only factor that showed a meaningful correlation with the size of damages awarded by synthetic juries (those composed of test subjects that were asked to decide damages under a number of different scenarios) was with whether or not a risk assessment was performed.
The personal characteristics of jurors didn’t matter. The cost per life saved didn’t matter. Even a high absolute level of risk didn’t matter. The only factor that was significant was whether or not a risk assessment was performed.
Here has been no research similar to Viscusi’s that asks about damages from data breaches or other security incidents, but the fact that jurors might be offended by a careful risk assessment should be chilling to people in information security organizations. Without a risk assessment, you may not spend your budget in a reasonable way, but with one, you may be leaving yourself open to other complications.