threat hunting best practices

Incident Response Q&A Part II: Why Incident Response Playbooks Aren’t the Answer, and Advice for New CISOs

Incident response will continue to be an important cyber security priority for many organizations in 2018. We took a moment to get some deeper insight into the incident response landscape from Delta Risk Senior Consultant Ryan Clancy.

Here’s part II of our incident response discussion (you can find part I here).

Dev: There have been some discussions in incident response circles to move away from traditional incident response playbooks. Advocates for data science and automation have emerged. What are your thoughts?

Ryan: I am a fan of data science and automation approaches. Here’s why I’m not a huge fan of playbooks. They take a lot of time to create and get outdated very quickly. Any time you get a new employee, a new process, or a new technology, your playbook becomes out of date, and you have to make updates. Who likes making those updates? Who likes going back and updating documentation?

An alternative solution that isn’t so maintenance-heavy would be a welcome change. I’m not totally anti-playbook but playbooks have a lot of limitations. The playbook strategy can be significantly improved. Playbooks take a long time to get right and they’re not always used in practice. You would think it’d be easy to make updates, but it never ends up going that way.

Dev: When it comes to updating playbooks, is the problem that there are usually too many people giving input? Would hiring a documentation manager streamline the process?

Ryan: One of the challenges of hiring knowledge managers or document managers is they don’t always know what’s being said in the room and don’t always know what’s important to capture. They can certainly learn those skills over time, but they may not have those skills in the beginning.

Another problem with playbooks is that they never cover every case. For example, just looking at recent events in the news, there have been complex breaches, involving multiple layers of exploitations. It’s fine that one playbook won’t cover every step of the process, however odds are that there will be at least a remediative step that you won’t have a playbook for.

Also, a lot of compromises are based on stolen credentials. You can’t really follow a cyber security incident playbook to find a solution to those compromises. If credentials are stolen, you’d reset the credentials – you wouldn’t need a playbook to tell you that.

The subsequent steps to ensure the credentials aren’t stolen again vary so greatly it would be difficult to create a playbook for each of them.

Ultimately, I find playbooks either way overkill or way underkill. I’d been more playbook optimistic if there were documentation systems that provided more sturdy support of the velocity of changes that occur in security.

Dev: Shifting gears a bit, incident response directly impacts CISOs. If you were to advise a new CISO on how to handle incident response program development, what would you recommend?

Ryan: I would offer a couple of pieces of advice. If you’re starting a new program, pick the right people for your team, specifically people who are response-minded. It’s also important to practice, practice, practice. Exercise internally. Here’s why: For a lot of CISOs, when an incident occurs, it’s the first time they’ve had to handle a challenge of that scale. It would be like asking someone who’s only ridden a bike to suddenly drive a NASCAR race car.

That’s why practice is so important. It can simply be an internal exercise program and they don’t have to be complicated exercises. You can spend an hour or two with your team, go through scenarios, and walk them through the steps of your process. Yours will vary, but it should start with rounding up all the information you can find. Then you need to make a decision based on that information and classify the incident appropriately. From that point forward, you should notify senior managers or senior reviewers of some kind. Once you have your team together, you can go forward to remediate the incident.

It’s important to practice all these initial steps and make sure you document the main points in your incident response plan. There’s a difference between a plan and a playbook – a playbook works within the overall incident response plan to distinguish key response tactics.

Dev: Thanks for those helpful tips. Alright, now that it’s officially 2018, do you have any incident response predictions to share with us for the new year?

Ryan: Sure, so I think prevention has turned into the red-headed stepchild of incident response, but it’ll be an important trend going forward. Nobody really wants to talk about prevention; it’s all about response. While I see a compelling case for using automation and data science for response, I also see a strong case for using automation and data science to improve prevention.

I’ll give you an example. If you have a lot of violations of removal media policies, it’s probably time to implement a technical control to stop those violations. If you have a lot of incidents focused on web use, or you’re getting your brains phished out, it’s time to start conducting prevention measures based on the historical data that’s available.

Companies have tried to play whack-a-mole to resolve incidents. You need to gather some metrics on how you’re getting attacked. NIST calls this a vector. You need to figure out what vector the adversary is using to attack you and how they’re delivering those exploits. Let’s stop these attempts earlier in the attack chain.

Companies now have the data to figure out these preventative steps. In the past, they weren’t really tracking their incidents. However, now there are more systems in place to analyze the data. By 2019 (if not 2018), you’ll see some rich data sets emerge to determine the trends.

Previously, Excel spreadsheets were the go-to method to gather metrics. Spreadsheets can be difficult to update and don’t provide quick answers. As automated technologies are built-in to your incident response toolset, the process to gather metrics can be simplified and less labor-intensive.

Organizations have been relying a lot on people to deliver metrics, but we need to turn to technology to handle these jobs. People don’t like doing manual, labor-intensive tasks. If you can rely more on software to handle the tasks on the backend of your incident response ticketing system, it’ll be an efficient way to go.

Dev: Relying on your people is not only manually-intensive, but it’s also an error-prone solution. For a successful incident response program, it looks like you have to determine the right moments when you rely on your people instead of relying on technology.

Ryan: Right, and exclusively relying on technology over people or vice versa is another mistake companies make when it comes to incident response. Sometimes they rely too heavily on technology to save the day. Don’t remove the human decision-maker from the process. As long as the information is valid, a human decision-maker is going to make the best decision possible based on an understanding of your business priorities. In comparison, software or technology isn’t going to be make a decision based on your business priorities at this point. Maybe we’ll be there someday!

Ultimately, incident response is a business capability. It’s a business capability you must adopt to protect your services or products in the business.

The truth is, no plan is going to be a substitute for good judgment. Maybe we’ll get there in a decade or two, but we’re not there now. At no point in the next few years will humans be able to take a step back and let artificial intelligence take over for incident response without some unforeseen consequences.

To learn more about incident response program development, check out our guide, “Best Practices for Integrating Incident Response and Business Continuity Programs.”