2008 Dubious Data Awards
Information security is a field in which it’s often frustratingly difficult to get accurate and useful data. Much of the data that is widely quoted isn’t actually accurate. We’ve mentioned this before when we talked about the misperceptions around the insider threat and the cost of managing passwords. And even if you have accurate data, the fact that technology changes rapidly means that the field of information security also changes rapidly as new vulnerabilities are discovered and old ones patched.
It looks like having statistics that aren’t quite accurate isn’t limited to just information security. There are enough cases of the improper or misleading use of statistics to justify a group of researchers forming STATS, a non-profit, non-partisan research organization affiliated with George Mason University. The mission of STATS is to point out the common misuse and abuse of statistics. The STATS web site has both quick summaries that describe cases where statistics aren’t used quite as accurately as they could be as well as longer case studies. Both are interesting reading.
STATS just released the 2008 installment of their Dubious Data Awards. None of this year’s awards seem to relate to information security, but they’re still interesting to read about. We should certainly be skeptical of any data that’s used to support either the sale of a product or a public policy decision. In both of these cases, there’s little incentive to present a balanced, unbiased view of what’s really what.