Organizations log thousands of security-related events every day from an array of devices and software. So, what’s the best way to go about managing your security logs? The reality is, most of the time, no one looks at these logs unless they have a security incident or data breach.
Some security professionals suggest that if you’re not actively monitoring events, then there is really no reason to log. While this is a hardline approach, there is some truth in it.
Most industries are governed by laws or regulations that require companies to log certain events and perform periodic reviews. These are also known as security audits. The reality is that many organizations meet the letter of the law and can check the box to satisfy auditory requirements. However, most fall short in actionable monitoring. There’s just too much activity to keep pace.
What is a Security Event?
Before we go further, what qualifies as a security-related event? The list is long, but some of the basic events are password changes, failed logons, or failed accesses related to information systems, administrative privilege usage, smartcard credential usage, or third-party credential usage, just to name a few. You can see how this can easily add up to hundreds, thousands, and hundreds of thousands – potentially millions – of events, even for a mid-size company.
Checking the Right Boxes
If you’ve worked in IT for any length of time, you’ve no doubt seen this. The regulation or guidance says “log it,” so you log it. Then it says “centralize all logs,” so you do that. If the budget allows, you buy a security information and event management (SIEM) to run all the heavy analytics of those logs. That way, you don’t have to hire people to review them, or to create custom scripts to go through potentially millions of records looking for something that happened days or weeks ago, right?
After all, the SIEM provides real-time analytics and will fire alerts and trigger alarms all day long. All you have to do is analyze each alarm and decide if it’s a valid or a false positive security-related event, and then take action accordingly. How hard could it be, right?
What Happens Next?
What happens next? The alerts and alarms start firing. Mailboxes fill up. You triage the first few hundred and figure out that they are false positives. The out-of-the-box configuration was good enough to include hundreds of the most common alerts by default, but they don’t really work for your environment. So, you create a rule to move all your alerts to a different folder in your mailbox to eliminate the noise. In that very moment, though, the solution becomes useless.
You look at this folder from time to time, but the alerts number in the thousands now. It’s not very useful anymore and basically, nobody is watching. But that’s okay! All of the requirements are in place to check a half dozen or more boxes on security controls to satisfy regulatory requirements and get your organization a good score on its next audit.
Your organization could be satisfying one or more of the more popular controls catalogs such as National Institute of Standards and Technology (NIST) 800-53, Payment Card Industry Data Security Standard (PCI-DSS), or Control Objectives for Information and Related Technology (COBIT). It’s easy to satisfy the controls, but actually putting them into practice is far more involved. Security events need to be captured 100 percent from all devices. This is a daunting task for most organizations.
Why Should You Centralize Logs?
Once the events are captured, they need to be centralized. This is typically done on one or more servers allocated to collecting logs. Some controls might not specifically say that logs must be centralized, but it is considered as best practice for several reasons.
First, it makes it a lot easier for administrators to manage the logs. If they need to be retained for any length of time, having the logs in one location makes backups and archival much simpler.
Second, event correlation. Correlating a sequence of events across an enterprise is extremely difficult and time consuming. Having them all in one location saves time. This is where a SIEM comes in. The SIEM automates much of the parsing and correlation activity, freeing up personnel to operate and maintain other hardware and tools more efficiently.
The Downside of SIEMs
A single event can generate multiple records in a log file, and there are multiple logs coming from every direction; firewalls, applications, servers, intrusion detection systems, and network devices – thus the hundreds of thousands of alerts we alluded to earlier.
Without a SIEM, it’s nearly impossible to review all of these log files and records. However, the reality is that SIEMs require continuous tuning to make sure relevant events are being captured and new events are included. Out-of-the-box SIEM configurations only capture the basics. You must take additional steps to tune them to meet the specific environment you’re in, along with any regulatory requirements.
Those basic alarms may be false positives in your environment and need to be tuned out, while things that should be triggering alarms go unnoticed. You need to fine-tune them during normal hours to learn what’s noise on the wire and only capture important events.
Most vendors provide regular updates to SIEM filters to help capture and trigger newly-discovered events. But even those vendor-supplied filters could trigger false positive alerts depending on the environment. The burden is still on the administrator to analyze SIEM alerts and periodically cross-check the logs in an ad-hoc fashion to validate the activity.
The term “eyes on glass” is thrown around a lot in the security world, but it’s a necessity. Organizations invest hundreds of thousands of dollars purchasing hardware and SIEM solutions, storage, and solutions to parse and analyze the events, but then don’t provide resources for the day-to-day care and feeding of the tools. The SIEM must be constantly checked against the logs to ensure it is capturing important events. The SIEM doesn’t “know” the environment as well as the people managing it, so it’s up to them to teach the SIEM what’s important and what’s not.
The bigger issue for many small and mid-sized organizations (SMBs) with smaller budgets and smaller security teams, however, is that SIEMs can be cost-prohibitive, time-consuming, and difficult to manage without experienced staff to run them. In addition to purchasing the actual SIEM, support and licensing costs can also add up very quickly.
Summary
As you can see, managing your security logs is more than just set and forget. There are a lot of requirements to keep up with, and it’s always a work in progress. Regulatory guidelines usually mandate the types of events to be logged as well as where the logs get stored and for how long. If for some reason you lack formal guidance, refer to the NIST security controls. If you’re considering investing in a SIEM, keep in mind that it won’t solve the problem of monitoring all the logs – it only makes your administrator’s job easier. You’ll need to continuously tune the SIEM to your specific environment to make monitoring more useful and create actionable activities, too, and budget for ongoing support and licensing costs.
One solution that many SMBs have found to deal with the avalanche of logs and alerts is working with a managed security services provider like Delta Risk instead. With a SOC-as-a-Service solution, organizations can offload all the work of managing and monitoring logs in house or investing in a SIEM. They get a team of experts available 24×7 who use advanced Security Orchestration, Automation and Response (SOAR) tools to analyze the volume of logs, all for a cost-effective, monthly rate.