Recently disclosed details about the Equifax data breach indicate that, in addition to the Apache Struts vulnerability that initially led to the breach, some security tools had stopped working for an extended period of time, and only after those tools were brought back online was the breach detected.
There are many potential reasons for security technology to fail, but quite often we don’t recognize that they have failed, because we are only monitoring for alerts, or simply assuming that the security “thing” is quietly doing its job. When those technologies do fail and we’re not aware that they’ve failed, a key element of our security program is not working, and if we’re not actively monitoring for those failures, we don’t know that we’re blind or unprotected.
For this reason, I recommend developing a set of ongoing test cases that are implemented along with new security technology to help ensure that the technology is operating as expected and raise an alert when it fails in some way. For example, a SIEM should be configured to trigger an alert if a log source does not provide a log within a certain timeframe, which may indicate that the logging service died on the host, or some network issue is preventing logs from being sent to the SIEM. Another example might be a periodic injection of a particular type of network “attack” (in a relatively safe manner, of course) designed to trigger an IPS block and alert, in a manner that tests both the blocking (did the “attack” make it to the destination?) and the alerting (did the “attack” result in a generated alert?).
These test cases should be developed to measure the ongoing effectiveness of all the key functionality that the security technology provides.