The field of information security is a prime example of making decisions under uncertainty. Generally, there is far more to do than can be done, and therefore we must make priority decisions of what to protect, where to invest, how and who to train, and so on. We know that we cannot create a perfectly secure system that retains some useful business value, beyond that of a doorstop or paperweight.
I recently started listening to Annie Duke’s book “Thinking in Bets: Making smarter decisions when you don’t have all the facts”. Like others in the field of behavioral economics, Mrs. Duke cautions us against the phenomenon of outcome bias and hindsight bias. Basically, we should reward good process, not good outcomes that might be/likely are the result of pure luck. This quote particularly resonated with me:
“An unwanted result doesn’t make our decision wrong if we thought about the alternatives and probabilities in advance and allocated our resources accordingly.”
This, by the way, is why we should be wary of management teams that purport to be “outcome based”. This means that management will almost certainly value luck over sound decision making. As with any random process, a “lucky” executive that was just promoted will very likely come to understand the concept of “regression to the mean”.
In the IT security world, we shame people and organizations that have a breach. We look at what happened with the benefit of hindsight and conclude that any reasonable person could have foreseen the breach, therefore condemning the person or organization as incompetent or negligent. Oddly, though, we (often) don’t know whether any of us could have made better decisions than the person or organization involved in a breach. Possibly the breached organization made reasonable priority and investment decisions but got unlucky. Or maybe the organization made crappy decisions and the breach really was inevitable. We never get to consider the contra scenario, where an organization makes crappy security decisions but gets lucky.
Luck shouldn’t be part of our strategy to defend the assets we are charged with protecting, but good process should. By that logic, we should direct our criticisms at organizations that have bad processes, whether they were breached or not.
Hopefully this is unsurprising and intuitive, at least upon hearing it. There are two problems with applying this concept to infosec:
- There is no objective way to tell whether a breached organization fell victim to crappy planning or had good planning and got unlucky. We can’t rely on the organization itself to help us out there. In some rare cases, we do get to make informed decisions based on the civil or criminal court proceedings.
- Unlike a poker player, as in Mrs. Duke’s book, who gets unlucky during a card game, when modern organizations are unlucky enough to suffer a breach, it is not just the organization itself that is harmed. Quite often, those harmed are customers, business partners, employees, and others.
For those of us who are harmed in data breaches through no fault of our own, we can’t simply accept that the breach organization was just “unlucky”. We believe that the breach happened is evidence that the organization was not doing enough to protect its systems. This gets to the heart of fundamental philosophical issue facing organizations in the age of pervasive data: unlike almost any other business risk that an organization faces, the harm from many breaches are not borne by the organization itself. Organizations are playing poker both with its own chips, and with the chips of the people it stores data on.
Regardless, organizations do not have perfect visibility into threats, nor do they have unlimited budgets, and so long as they handle such data, they will be making decisions on how to protect the data. Some organizations will find that items that fell “below the line” on the priority list created gaps that lead to a breach. Others will get lucky.
Laws like the GDPR will help, because the GDPR raises a possible significant fine and civil liability from not properly protecting personal data. I am skeptical that we will see any noticeable decline in data breaches after the law takes effect, because at the end of the day:
- We do not have perfect security
- Breaches are the result of effectively random processes
Wow…. I am a big fan of your podcast and came across a recent podcast called CBE talent angle where she breaks down the parallels between decisions and playing poker…. and I was thinking about your podcast… does this mean I am getting smart like u….
Please work this into the podcast…thanks for being on the internet … this is so informative. If only you understood how much your listeners enjoy your work.
I just looked up that podcast and it appears the interview is with the person who wrote the book I am reading. Nice find.
Also, I would bet you were already smarter than me.