In today’s blog, we’ll explain what Elasticsearch is and how it helps power comprehensive cyber security solutions you can use for your organization.
In some of our previous blogs, we’ve talked about our cloud-native Security Orchestration and Automation Response (SOAR) platform, ActiveEye. Part of the muscle behind this platform is Elasticsearch. Elasticsearch makes volumes of data usable in real-time and at scale. It can be used to power faster searches, a search engine, enterprise search capabilities, logging, security, and business analytics, for example.
Unlike similar tools, it can be easily expanded and provisioned for faster searches. It’s part of the Elastic Stack, which stores data centrally. The other components of Elastic Stack are Beats to ingest data, Logstash to normalize data, and Kibana, a dashboard to visualize it all.
What’s the Secret Behind Elasticsearch?
One of the features that allows Elasticsearch to move so quickly and scale either up or down with an organization is the JSON Document. JavaScript Object Notation (JSON) is an open-standard file format that uses human-readable text to transmit data objects consisting of attribute–value pairs and array data types. The standard organization of JSON documents makes searching easier, as data is all in the same format. All the fields in the documents are also aligned, which allows you to target the exact search fields you need. This makes searches significantly faster.
While the “J” in JSON stands for JavaScript, JSON is a language-independent data format. Most modern programming languages include code to generate and parse JSON-format data. The Logstash function in Elasticsearch does that. When data is ingested into Elasticsearch, it is normalized and indexed using the JSON schema.
Once the data processing is done, you can create complex queries to run against it. You can also use aggregation capabilities to retrieve complex summaries of the data. With the data in the common schema, the complexity of the search can easily increase, without a corresponding increase in search time. The aggregation functions, like sum, average and count, are also faster, as they key into the specific fields needed. Additionally, the scaling feature allows you to tailor full text searches and storage requirements. Elasticsearch is available in the cloud or as an on-premises solution. Either can add capacity easily.
Why Is Speed Important?
Response times in cyber security incident recovery are key. Attackers can do damage almost instantly if they get into your environment, and Elasticsearch is built for speed. Our ActiveEye 2.0 platform relies heavily on technology like this to power security automation, which is critical to identifying threats and resolving alerts faster.
In ActiveEye, you can see an exact timeline in your personalized dashboard that shows when a cyber security alert is created and when it’s resolved. These dashboards are built on Kibana.
Elasticsearch Adds Endpoint Security Capabilities
At the 2019 Elastic conference, Elasticsearch announced that it had acquired endpoint security company Endgame. According to MSSP Alert, the deal will blend Elastic’s SIEM (security information and event management) with Endgame’s endpoint prevention, detection and response capabilities. Similar to next-gen endpoint solutions like Carbon Black and CrowdStrike, Endgame enables security professionals to better prevent threats like ransomware and phishing.
The timing of the announcement is crucial. As many businesses move towards a global and remote workforce, the number of endpoint devices continues to grow exponentially. Additionally, technology is constantly changing. Those factors can make endpoint security a nightmare for traditional IT departments to manage. It can also complicate decision making for Chief Information Security Officers (CISOs) and Chief Information Officers (CIOs).
Elasticsearch in Public Sector
Recently, the Department of Homeland Security, in conjunction with the General Services Administration (GSA), announced that it will use Elasticsearch for its next-generation Continuous Diagnostics and Mitigation (CDM) Dashboard Ecosystem. The CDM Dashboard Ecosystem gives federal agencies a much needed cyber situational awareness solution. They can visually see the status of their agency’s devices and associated cyber risks. They can also manage access for whoever’s using their network. The dashboard shows what is on the network and who is on the network. Additionally, the dashboard summarizes risk exposure across the government at the federal level for DHS.
This change has wide-ranging implications for all federal agencies that use the DHS-provided solution. The magnitude of cyber security risks – coupled with the growing number of devices that U.S. government agencies use for daily operations – demands a responsive, scalable, and fast solution.
The dashboard will include ongoing advancements in data visualization, data ingesting, indexing, and search performance. Cyber security analytics will be delivered using business intelligence, artificial intelligence, and machine learning technologies, all stored in Elasticsearch.
The government intends to integrate future capabilities into the dashboard to include a threat intelligence platform for government-wide threat indicator sharing. This will include integration of a risk-scoring algorithm, and security orchestration automation and response (SOAR) tools for key vulnerability remediation and defensive cyber operations.
Summary
At Delta Risk we’re excited to use Elasticsearch as part of our ActiveEye cyber security platform. The speed and scalability of the solution has allowed us to grow with our customer base, while constantly improving the response time to cyber security alerts and incidents. It remains to be seen exactly how Elasticsearch will integrate Endgame’s functionality. However, the upcoming launch of new endpoint security features is expected to add more value for IT and security professionals, as well as our security operations center (SOC) team.