Cyber Introspection: Look at the Damn Logs

I was talking to my good friend Bob today about whatever came of Dick Cheney’s weather machine when he interrupted with the following question:

“Why, as an community, are we constantly seeking better security technology when we aren’t using what we have?”

Bob conveyed the story of a beach response engagement he worked on for a customer involving a compromised application server.  The application hadn’t been patched in years and had numerous vulnerabilities for anyone with some inclination to exploit.  And exploited it was.  The server was compromised for months prior to being detected.

The malware dropped on the server for persistence and other activities was indeed sophisticated.  There was no obvious indication that the server had been compromised.  System logs were cleared from the time of the breach and subsequent logs had nothing related to the malicious activity on the system.

A look at the logs from a network IDS sensor which monitors the network connecting the server to the Internet showed nearly no alerts originating from that server until the suspected date of the intrusion, as determined by forensic analysis of the server.  On that day, the IDS engine started triggering many, many alerts as the server was attempting to perform different activities such as scanning other systems on the network.

But no one was watching the IDS alerts.

The discussion at the client quickly turned to new technologies to stop such attacks in the future and to allow fast reaction if another breach were to happen.

But no one talked about more fully leveraging the components already in place, like IDS logs.  IDS is an imperfect system that requires care and feeding (people); clearly an inferior option when compared to installing a fancy advanced attack.

I previously wrote a similar post a while back regarding AV logs.

Why are we so eager to purchase and deploy yet more security solutions, which are undoubtedly imperfect and also undoubtedly requires resources to manage, when we are often unable to get full leverage from the tools we already have running?

Maybe we should start by figuring out how to properly implement and manage our existing IT systems, infrastructure and applications.  And watch the damn logs.