In the infosec industry, much of the thought leadership, news, and analysis comes from organizations with something to sell. I do not believe these groups generally act with an intent to deceive, though we need to be on guard for data that can pollute and pervert our understanding of reality. Two recent infosec-related posts caught my attention that, in my view, warrant a discussion. First is a story about a study that indicates data breaches affect stock prices in the long run.
Here is the story: https://www.zdnet.com/article/data-breaches-affect-stock-performance-in-the-long-run-study-finds/
Here is the study: https://www.comparitech.com/blog/information-security/data-breach-share-price-2018/
Most of us who work in the security world struggle to justify the importance of new and continued investment and focus on IT security initiatives, and the prospect of a direct linkage between data breaches and stock price declines is a wonderful thing to include in our powerpoint presentations. As humans, we are tuned to look for information that confirms our views of the world, and the results of this study seem intuitively correct to most of us. We WANT this study to be true.
But as with so many things in this world, it’s really not true. To the credit of the study’s authors, the study includes a section on the limitations of the study, but that really doesn’t detract from the headline, does it? So, I propose an alternate headline: “Data Breach proves to be a boon for LNKD shareholders!”.
In addition to the issues identified in the “limitations” section, there are other confounding factors to consider:
- The all had data breaches. I know that sounds dull, but consider running a study of people who lost weight, and only including in the study people who are leaving a local gym in the evening. Do companies that experience data breaches have some other attributes in common, such weak leadership or having a culture of accepting too many risks? Might these factors also manifest themselves in bad decisions in other aspects of the business that might result in a declining stock price? We don’t actually know, because the only way to know for sure is through experiments that would be highly unethical, even if immensely fun.
- Averages don’t work very well for small data sets. Consider the following situation
- Company A, B, C, and D all suffer a data breach on the same day
- Company A, B, and C all see their stock rise by 2% the week after their respective breaches
- Company D sees it stick decline by 20% the week after its breach
- The average decline for this group of companies is 6.5% the week after their beaches. But that doesn’t tell the whole story, does it?
I’m not saying that breaches don’t cause stock prices to decline. I am saying that I’ve not yet seen good evidence for that, and that is because we can’t fork the universe and run experiments on alternate realities and compare the results. If we could, this would not be among the first experiments I’d propose.
Like a good Ponemon study, this is study is great fodder for executive meetings, but be ware that you are not on firm ground if you get challenged. As an anecdote, I used to be a pretty active investor, and while I did not get the Ferrari, I did learn a few things:
- I am apparently supposed to buy low and sell high, not the other way around
- Breaches, from a pure inventor standpoint, are generally viewed as a one time charge, and (generally) do not change the underlying fundamentals of the company. When investing in a company, it’s the fundamentals that matter, such as: are their sales going up and cost of sales going down?
Next, is a story about a study that indicates 90% of retailers “fail PCI”.
Here is the story: https://www.infosecurity-magazine.com/news/over-90-of-us-retailers-fail-pci/
Here is the study: https://explore.securityscorecard.com/rs/797-BFK-857/images/2018-Retail-Cybersecurity-Report.pdf
Unfortunately, the authors of this report don’t give a description of the limitations, but I think we can infer a lot about the limitations based on the type of testing this organization performs to gather the data. That company gathers and collates open source intelligence, seemingly similar to what other players like BitSight are doing. I would assert that the report finds that retailers are among the worst industries, based on the data this organization gathered, at patch management. Without knowing the details of each company in the study, we can’t know whether the environment analyzed was part of the PCI DSS Cardholder Data Environment (CDE) for a given retailer. Making an assertion that an organization who seemingly must comply with PCI DSS is violating their obligations based on a review of the organizations “digital footprint” is not appropriate. I am not defending the organizations’ lack of patching, just that patching all of an organization’s systems is not a PCI DSS requirement, though maybe it should be.
The downside in this sort of report is that it likely “normalizes” non-compliance with PCI-DSS. If I’m spending a tremendous amount of time, energy and money to keep my environment in the right shape for PCI, but then see that 90% of others in my sector are not doing this, how motivated with I or my management team be? The “right” thing to do clearly doesn’t change, but this study changes our perception of what is going on in the world.
I had a math teacher in high school who told us to keep an open mind, but not so open that people throw their trash in. Remember to maintain a healthy level of skepticism when reading infosec articles, reports, and studies… And yes, even blog posts like this one.