Does your business want to set up or improve its threat intelligence security operations? If you find it more complicated that you thought, you have plenty of company. Many companies wanting to benefit from TI are learning a hard truth: setting up successful TI operations takes a lot more than subscribing to a TI service and running data feeds through a TI platform. So, what’s in the “fine print” of setting up successful TI operations?
As we mentioned in a previous post, many must-have TI resources involve hardware, software and (for some businesses) cloud-based services. But, there are many good-to-have resources, too. Clearly defined business goals and people who can transform analytics results into security methods are just two examples.
Out of curiosity, we made a list of suggested business- and IT-side tasks and resources that go beyond the obvious hardware and software TI requirements. When the list passed 10 items, we knew we should share the information with you.
First up, business planning for your TI system.
The most surprising thing about this assignment was how important business-related thinking and planning was. The idea was to suggest thinking ahead, setting goals and connecting successful TI system functions with the business results your company wants.
Here are planning-related topics. Consider them when it’s time to set up or upgrade your company’s TI resources and methods.
Before you dive into the details of how you are going to use threat intelligence, it pays to have goals and then back into the capabilities you need to achieve those goals. This is not a new approach. But many analysts concentrate on system capabilities.
This is a mistake.
High-speed, high-volume data handling performance might please the IT folks, but value is the key to the hearts of the folks paying the bills. So, start with business value.
What do your CIO and CISO want to achieve in the long term? What do they want to avoid? For example, do they want:
- More efficient operations? If making TI more useful and manageable is the goal, look for TI platforms and services that reduce the time and effort needed to find, gather and analyze routine threat data.
- Faster, more accurate response to attacks? If reducing post-attack downtime and avoiding damage to your company brand are key, consider platforms or services that focus on rapid-response capabilities.
These are just examples. There are bound to be additional, value-related scenarios that are relevant to your security operations. But, answers to these questions provide general directions for your TI efforts. Setting goals to TI capabilities answers the question, “How well are we doing?”
You’ve identified the security tasks that are likely to deliver maximum value to your TI operations. Next step: setting clearly defined business goals for those operations.
 Approaching TI with clear operational goals influences your understanding of how and where intelligence can provide value to TI operations. Use these goals to customize your TI program and solve specific security problems.
Critical success factors (CSFs) and key performance indicators (KPIs) are the only way to identify what and how you want to measure TI ops behavior. Use these goals, (which should use bona fide units of measurement), when you:
- Define high-value security business goals.
- Identify TI operations process or task for each high-value business goal.
- Identify potential security operations problems for each process or task,
Next up: Identifying the human and IT resources you’ll need for successful TI operations.
IT-Side Resource Planning
Here are some IT-related considerations for TI program designers.
Successful TI is not a machine-only proposition. You’ll need human as well as IT assets to solve your security ops problems and meet your business goals. Consider the need for these members of your TI team:
- Security ops specialist. Translates results run through data analytics (and possibly an SIEM system) into relevant security processes and tasks.
- Security data analytics specialist. This individual knows how to set up data queries in high-volume data analytics programs and interpret pattern matching and machine learning results.
And next, some cures for some expensive data issues.
Getting to Know Your Data Sources
Organizations that engage in the plug-and-play school of TI quickly become overwhelmed by the sheer quantity of low-yield alerts these sources provide. That’s because the threat “intelligence” tends to be raw data, not information, let alone intelligence.
Relying purely on open-source data feeds sets your TI program up for “alert fatigue” and possibly for a lost opportunity. There are many different sources of threat data, each with its own advantages and drawbacks.
Best results (most relevant data hits and fewest false positives) occur when you automatically combine multiple data sources to confirm and customize threat information before you hand it off to a human analyst.
A Tale of Two Data Streams
Consider these scenarios:
This is a comprehensive list of steps that convert raw threat data to intelligence. Not all TI solutions have all these capabilities, but more and more solutions use them.
§ Combine many formats of data coming from many sources.
§ Send threat data to a single, centralized location (portal).
§ Use natural language processing to capture unusual speech patterns in data.
§ Use advanced data analytics to automatically perform pattern recognition tasks and convert raw data into an easily understood format.
§ Integrate TI and SIEM solutions. Analytics processes eliminate many false positives. SIEM provides the context necessary for a human analyst to triage security events up to 10 times more quickly than with manual methods.
- Scenario 1: your TI platform gathers huge amounts of raw TI data from many sources without your customizing the data feeds. Result: the data flow overwhelms human analysts with false positives.
- Scenario 2: Data feeds are customized to reflect the unique security operations of your business. Result: The value of your threat information soars, and the number of false positives plummets.
The remarkable difference in these scenarios relies on data processing software and methods and on your choice of data sources.
Don’t Analyze Irrelevant Data
Answering these questions will help you avoid wasting time and resources analyzing irrelevant data:
- Think back to your business goals and what you want to occur or avoid.
What mix of open-source and premium data feeds do you need to analyze to meet your goals?
- How many and which types of data feeds does your system use?
- Where do your data feeds come from?
- How much of the data they provide applies to a company in your industry or sector?
- How will your TI provider deliver data?
If you rely on hosted TI, your provider will offer it as a premium threat feed, a pre-
packaged software product, or as a single-customer report.
Check to see whether your provider uses a mix of human and automated security operations, and harvests intelligence from open and closed data sources. Is this the mix that will help you achieve your goals?
Organizing TI Data and Security Ops
Before you can correlate, enrich and customize TI data, you must organize it. Here are a few ways to organize the data and its users.
- Create a destination for all TI data. Centralizing TI data removes barriers between silos of internal and external data sources. And, it makes it easier to find, clean up the data and transforming everything to a single data model.
- Create a common workspace for stakeholder teams. Having a centralized place for executives, TI, risk and security ops specialists to create and share their work is a proven way to operate more efficiently.
- Use or create TI playbooks. Guide your intel processes by creating or assessing intelligence methods based on internal and external sources.
Reviewing TI Platforms and Services
When you assess TI platform capabilities, consider this: The goal of TI platforms is a balance:
Receive and analyze enough data to improve the likelihood of receiving alerts your organization needs to remain secure
being overwhelmed by irrelevant data and false positives.
When you sign up for a TI platform or service, what do you get? Capabilities. That’s the nitty-gritty. You pay for the ability to do specific security-related tasks with specific types of data and tools.
What is TI?
Threat intelligence is not just any platform output. It is output that
§ Relates to your security ops (relevant).
§ Put into context of your SO (contextualized)
§ Can be used in specific security functions such as processes, best practices, etc. (operationalized).
§ Is easily understood and used.
Here’s a list of basic TI capabilities:
- Collects data from many of feeds and brings it to a single location.
- Receives alerts in real time.
- Normalizes feed data (remove duplicates, enables user-set rules, etc.)
- Integrates with SIEM, firewall logs, etc.
- Creates reports
- Provide actionable indicators used to identify potential threats
- Identify and contain new attacks automatically
- Analyzes security data automatically
- Integrates TI capabilities with other security tools.
Kicking TI Capabilities UP a Notch
These more advanced capabilities are also included in TI platforms and services:
Big data analytics capabilities. This enables organizations to collect and customize a very large quantity of potentially valuable data and separate useful and irrelevant data automatically.
The ability to integrate data analytics and existing SIEM solutions. The analytics results are handed off to the SIEM, which now processes far fewer false positives. This enables human analysts to look at and respond to far fewer potential threats more quickly.
Use machine learning to recognize threat patterns. TI solutions produce genuine threat intelligence by using machine learning capabilities to discover, combine and contras threat data and information from a broad range of external and internal sources.
The key thoughts here are “faster” and “more relevant.” The very best threat intelligence solutions can compare your TI alerts with other data sources, internal telemetry, and a detailed understanding of your organization’s IT infrastructure.
Machine Learning Capabilities
Although most of the AI-related claims that TI vendors make is futuristic hoo-hah, machine learning has a modest place [Add link to title of Article 7] in security operations.
We mention machine learning here because security threat response time is a big concern among CSOs and security professionals. The longer something goes undetected, the greater the risk of lost revenue, employee productivity and damage to a company’s brand.
Advanced TI systems include data analytics in machine learning scenarios. Analytics solutions scan a wide range of external intelligence sources to identify security system baselines, outliers and anomalies. Analysts who use analytics in TI systems process data (firewall logs, for example) 10 times faster than the manual inspection of alerts.
Drowning in a Tidal Wave
73% of enterprises surveyed say they ignore security events because they’re overcome by a deluge of alerts.
Enterprise Strategy Group report (201x)
Adding Big Data Analytics to Threat Intel
A deluge of data is drowning security teams, who must sift, separate, and correlate the real threats from the false positives and irrelevant information.
Around 30% of survey respondents say they analyze about 11 different threat intelligence feeds. This overwhelming surge of information isn’t much help if security pros can’t prioritize and use information intelligently.
Algorithms used in large-scale business intel data processing are familiar tools. All analysts have to do is use frameworks such as Apache Hadoop and inexpensive, industry-standard servers and storage hardware. Voila! A high-speed, high-volume analytics solution is born.
TI vendors now add big data analytics capabilities to TI systems to filter indicators of compromise (IOCs) and other threat information for security event and information management (SIEM) systems, which weren’t built to process millions of IOCs.
Click to download your free workbook of questions to help your business organize successful TI operations.