Browsing Tag

cyber threat intelligence

Structuring Cyber Threat Intelligence Assessments: Musings and Recommendations

January 15, 2022

Tomorrow (Jan 16th 2022) I’ll be speaking at the free virtual conference PancakesCon 3 on “Structuring Intelligence Assessments and Gin Cocktails.” The conference’s format is to introduce material to students or new folks to a topic in the first half of the presentation and in the second half do something completely not work related, my second half will be non-intimidating and easy yet delicious to make gin cocktails (normally I’m more of a bourbon/rye guy but when it comes to cocktails I’m a gin fanatic).

As a result I decided to sit down today and write about the first half of my talk while practicing the second half of the talk. Therefore this blog will likely be more musings and recommendations than wildly coherent and well written thoughts. The talk is only 15 min long so I want to condense as much as possible into that timeframe and map the blog accordingly.

If this blog gets your insights going and you find the topic fun you can check out my SANS FOR578 – Cyber Threat Intelligence class where we talk about these topics and more. I’ll also include some reading materials and references at the end of the blog as well. Unfortunately and fortunately this one topic could (and has throughout history) been a complete manuscript/book but I’ll try to give me abbreviated version on the topic.

First Key Point

There’s a lot of opinions as it relates to cyber threat intelligence. I often semi-joke that the only thing Analyst 1 and Analyst 2 can agree on is that Analyst 3 is wrong. Intelligence doesn’t specialize in the area of facts. If everything was a fact or a simple yes or no answer you likely wouldn’t need intelligence analysts. Intelligence analysts specialize in going into the unknown, synthesizing the available information, and coming up with an analytical judgement often requiring you to go beyond the evidence (an analytical leap). There are some things you shouldn’t do, but generally speaking my retort to people is: “you do you.” There is no one right or wrong answer. I’ll give you my thoughts around intelligence assessments in this blog to help you but it should not be construed as the only answer. It is simply a consideration built on experience. The first rule of cyber threat intelligence, to me, is that if you’re being an honest broker then so long as you’re satisfying the requirement of your customer then everyone else’s opinion is irrelevant. Or said differently: everyone’s a critic, if you are delivering intelligence to the best of your ability to a consumer against their requirement and your intelligence helps them achieve their outcome then all the nit picking in the world about how you got there is irrelevant to me. Key focus on honest though. We deal in the world of trust because a lot of what cyber threat intelligence analysts do looks like magic to those not in our field. If you betray that trust, especially on topics that people aren’t well versed in and thus need to rely on trust more, then just pack up your bags and go home – you’re done. (As an aside that’s why I hate those stupid cyber attack pew pew maps so much. I don’t care how useful they are to your budget or not – it’s dishonest. The visualization is pointless and if a consumer digs a layer deeper and finds out it’s just for show then you come off as someone that is willing to mislead them for a good outcome. I.e. you lose trust).

Being Precise In Word Usage

Much has been written about intelligence assessments. One of the required readings is by Sherman Kent from 1981 at the Central Intelligence Agency titled “Words of Estimative Probability”. One of the beautiful things that Kent puts forward is the necessity to be measured and consistent in what we do. I.e. it’s not paramount to pick one direction over another but it is paramount to be transparent and repeatable in what direction you chose and consistent in its application. If you decide that you like the word “even chance” then you should know what that means to you (50/50 probability subjectively but as appropriately applied as you can) and consistently use that language in those scenario. You are more than welcome to use other people’s standards (like Kent’s) or create your own. It’s likely better to use what exists, but if you need to adapt or create your own you can just make sure it’s something you can make available to your team and consumers so its transparent and has definable meaning. I would recommend creating a style guide for you/your team and defining the words you do and do not want to use and as much as possible tying it to a number to help them understand how those words relate to each other. Defining words you do not use is also helpful. As an example, I cannot stand the word “believe” in intelligence assessments. As my friend Sergio Caltagirone would say belief is for religion not intelligence.


Figure 1: Example from Kent’s Words of Estimative Probability

One of the most important things intelligence analysts can do is reduce the barriers and friction between a consumer consuming the intelligence and leveraging it. Write briefly. Don’t use more words than required to make the point. Remove ambiguity wherever possible. In this topic, to convey what we do and don’t know as quickly as possible but in such a way that doesn’t require the consumer to learn our language over and over. The consumer should be familiar over time with the language we use and what it means to help inform them how and when they can leverage it against their requirement.

In a perfect world I want intelligence reports to be subjected to the Pepsi Challenge. I should be able to remove the logos, branding, color schema, etc. off of your intelligence product/report and put it in front of a consumer that you deal with and place it randomly next to other teams’ intelligence reports. The consumer should be able to pick out which report is yours. Language consistency, structure of the report, where the assessments are, where the suggested actions are, etc. all play into that.

Following that logic then our intelligence assessments should also be as consistent as possible.

Intelligence Assessments

If something is a yes or no answer then it is a factual statement. “Was this malware on this system?” can be answered as a yes or no question because the malware is either there or not and it is something we can prove. There should not be intelligence assessments for factual statements even if the factual statement is part of the threat intelligence. I.e. I would not expect to see someone say “We assess with moderate confidence the malware is on that system.” You can go and prove it or you cannot, but it’s not an assessment. The world of intelligence is for one step beyond the evidence. It’s the analysis and synthesis of the data and information to create new insights. However, we might say something like “We assess with low confidence, given the presence of the malware on the system, that our company is a target of their operation.” We do not know for sure whether or not the adversary intended to target us or if we were just a random victim. We can synthesize the available information and make an assessment based on our experience and what we’re seeing to reach an intelligence assessment.

I see most organizations leveraging Kent’s estimative language (or something close to it) in their wording and then using the estimative language you’d find in government intelligence agencies. Typically that is:

  • Low Confidence
  • Moderate Confidence
  • High Confidence

You can create middle grounds like Low-Moderate or Moderate-High but I try to avoid that myself as it often is more confusing to the consumer. Keeping it to three confidence levels in your intelligence assessment and having some rules you set out for yourself and your team on what that means often suffices.

General Rules for Confidence Levels

Again, you do you. But for me I like to generally follow the following guidance as it relates to designating confidence levels. Also I do think there’s a difference in cyber threat intelligence vs. the intelligence you might be delivering to the President of the United States to go to war. But these generally hold.

  • Low Confidence: A hypothesis that is supported with available information. The information is likely single sourced and there are known collection/information gaps. However, this is a good assessment that is supported. It may not be finished intelligence though and may not be appropriate to be the only factor in making a decision.
  • Moderate Confidence: A hypothesis that is supported with multiple pieces of available information and collection gaps are significantly reduced. The information may still be single sourced but there’s multiple pieces of data or information supporting this hypothesis. We have accounted for the collection/information gaps even if we haven’t been able to address all of them.
  • High Confidence: A hypothesis is supported by a predominant amount of the available data and information, it is supported through multiple sources, and the risk of collection gaps are all but eliminated. High confidence assessments are almost never single sourced. There will likely always be a collection gap even if we do not know what it is but we have accounted for everything possible and reduced the risk of that collection gap; i.e. even if we cannot get collection/information in a certain area it’s all but certain to not change the outcome of the assessment.

For purposes of clarification, the topic of “single sourced” to me relates to where we are getting the information. As an example, if you are operating only on netflow or malware repositories like VirusTotal it is highly unlikely you will get to a high confidence assessment out of any one of those. It is possible that you have commercial access to netflow, and malware repositories, and you have information shared from 3rd parties that help you get to a high confidence assessment. In a perfect world you’d have non-intrusion related information as well depending on the assessment. (e.g. you’re trying to attribute an operation to a specific government or government agency I’d prefer to see you have a lot of different types of 1st party data as well as 2nd or 3rd party data supporting the assessment and in an ideal scenario have more than just intrusion data. This is an area I often conflict with the private sector intelligence reporting; while I am a huge fan of private sector intelligence and think it runs circles around some types of government intelligence I generally hold a higher standard for confidence assessments on the topic of attribution than I see represented in *some* public reporting. That’s not a knock on every team out there – but I see “high confidence” for attribution thrown out quite a bit for data that just came out of an incident response case or malware repositories and you really need a lot more than that in my opinion).

Collection/Information gaps are anywhere there’s useful information or data that you don’t have access to. As an example, if an adversary targets an electric power plant I may have access to their malware but maybe I don’t have the initial infection vector. The missing data/information on the initial infection vector would be a collection gap. Additionally, maybe I see what their command and control server is but do not have access to what’s on the command and control server, who is accessing it when, etc. and those would be collection gaps. Some collection gaps you can solve for, some you cannot. But you must think through as many of them as possible and what that might mean to the assessment.

Structure of an Assessment

Again it really depends on you and your use-cases but generally speaking I like intelligence assessments’ structure to also be consistent and follow a repeatable and understandable pattern.

I often like my assessments to follow the following pattern:

Confidence + Analysis + Evidence + Sources

As an example:

“We assess with moderate confidence that ELECTRUM is targeting the Ukrainian electric sector based on intrusions observed at the Kyiv electric transmission substation by our incident response team as well as publicly available data from the Ukrainian SSU.”

The depth you go into analysis, evidence, and sources will vary entirely based on what your consumer needs and finds relevant.

Everyone Loves to be High

One of the big flaws I see in cyber threat intelligence teams is a well intentioned but misplaced effort to have as many assessments as possible be High Confidence. Your consumer may expect high confidence assessments. You may have accidentally trained them to expect that those are the ones to operate off of. But in reality, a low confidence assessment is a perfectly valid and well supported assessment. If you aren’t able to make an assessment with confidence levels its perfectly fine to say something to the effect of “We haven’t assessed the situation yet but here is my analytical judgement based on my experience and the available information.” In other words not everything needs to be an intelligence assessment.

But if you do give an intelligence assessment you need to train your consumer to understand that all three levels are appropriate to use. A low confidence assessment is a good assessment someone can have confidence in. Often times, the difference between a Low Confidence assessment and a High Confidence assessment comes down to time and sourcing. Do I have the time and collection to get to that level of confidence or not? While I love giving consumer’s Low Confidence assessments its perfectly reasonable to offer them what it would take to get to a higher level if they really need it: “If we had X more time and Y more resources we likely could raise the confidence level of our assessment or find an alternative assessment.” Be careful in how you use that but I generally find being transparent with consumers to empower them to make better decisions is almost always a good thing.

If we accept that low confidence assessments are good assessments. And we accept that to get to a Moderate or High level generally requires more time and resources. Then if were to look at all the assessments an intelligence team produces across a given time period (e.g. quarterly or annually) would fall into an inverted pyramid pattern.


Figure 2: Rob Makes a Pyramid in PowerPoint After Drinking Lots of Gin

I would expect to find that roughly 40-60% of the assessments produced by the team are Low Confidence Assessments. I would expect 20-30% are Moderate Confidence. I would expect 10-20% are High Confidence. That’s a general rule of thumb; many variables will impact that including the needs of the consumers. But generally speaking if I am trying to only release Moderate and High Confidence assessments the impact to my consumers is usually requiring more time and resources than necessary if they are comfortable making a decision on Low Confidence assessments. If the consumer truly requires a higher level of confidence – no problem! But if they don’t require that then don’t put barriers in between them and the intelligence they need to make a decision.

“But Rob my boss really only listens if we have High Confidence assessments.” I hear this often with my FOR578 students. You need to sit your consumer down then and have a conversation about this and how they can leverage your lower confidence assessments. You can try to pull historical information on your assessments vs. the decisions made and try to showcase that Low Confidence assessments are good assessments, they have a role. There’s tips/tricks here for this but it’s not really the point of the blog and I’ll just note that we serve as the request of the consumers (intelligence does not exist for intelligence’s purposes) but we want to make sure to arm them with the best information possible for their decision making process. Sometimes that does include correcting them.

Closing Thoughts

As mentioned this isn’t meant to be a doctoral thesis on the topic. This is largely just a resource for the people at my talk tomorrow and hopefully useful to others as well. However, there’s a lot of resources out there (some referenced in this blog) to give you help. Here’s some of my favorite as it relates to this topic:

The Cycle of Cyber Threat Intelligence
Presented by one of my awesome friends/peers/FOR578 instructors Katie Nickels where she highlights some of the FOR578 material in a masterful way to condense the view of cyber threat intelligence to folks in an hour webinar.

Hack the Reader: Writing Effective Threat Reports 
Presented by Lenny Zeltzer this is a great conference talk on threat reports that also touches on formatting (don’t roll your eyes, reduce friction for your consumers its part of the job)

Pen-to-Paper and the Finished Report
Presented by Christian Paredes this is a mini master level class on dealing with tough questions and translating things into useable intelligence reports.

Threat Intel for Everyone: Writing Like a Journalist to Produce Clear, Concise Reports
Presented by Selena Larson who I had the privilege to work with for awhile at Dragos. I would highly recommend any intel team look at hiring journalists to the team at some point.

I Can Haz Requirements? Requirements and CTI Program Success
Presented by Michael Rea, at the end of the day everything ties to the requirement. Do not do intel for intel’s point. No one cares how smart you are or how much you learned. They care about you giving them information that meets their requirement so that they can achieve their goals/outcomes. This is a great talk on that point that wraps into assessments as well.

 

*Edit Jan 16th 2022*

By popular demand here is my excel spreadsheet for Gin cocktails for my PancakesCon talk. I’ve changed some of the recipes a bit for my own personal taste. Feel free to adapt.

Gin Cocktails

Russian Election Meddling, GRIZZLYSTEPPE, and Bananas

August 17, 2017

It’s been awhile since I’ve been able to post to my blog (as it turns out doing a Series A raise for my company Dragos has been time consuming so I apologize for the absence in writing).  But it is fitting that my first blog post in awhile has something to do with the GRIZZLYSTEPPE report. I almost got sucked back into writing when I saw the Defense Intelligence Agency (DIA) tweet out the Norse cyber attack map.

Matt jumped on it pretty quickly though which was great.

I tried to attempt to fill the person in running the account just in case they didn’t understand why folks were less than excited about their presentation.

But in their responses to me it seemed they didn’t fully understand. They articulated that they use unclassified data for the conference but use classified data at work. Of course the problem wasn’t the data (even though it’s not just unclassified but completely bad/fake data) it’s the idea that a cyber attack map aka “pew pew map” is not a good way to communicate to any audience as its simply a marketing ploy. However, it’s not worth a full blog post so I’ll just instead request everyone to do their homework (should only be a quick Google search) on why pew pew maps are stupid and everyone serious in the discussion should stop using them.

On To the Main Discussion

But on to the main topic. What does Russian election meddling, the GRIZZLYSTEPPE report, and bananas all have in common? Absolutely nothing. Each are individually completely unrelated to each other and people should stop putting any of them together as it ultimately just makes people look silly (to be fair no one’s associated bananas with the election interference yet but it might be a better correlation than the GRIZZLYSTEPPE report).

This discussion was all spawned by an article that the New York Times released on August 16th, 2017 titled “In Ukraine, a Malware Expert Who Could Blow the Whistle on Russian Hacking“. Spoiler alert: he can’t. I went on a bit of a Twitter rant to explain why the article wasn’t good, it can be found here, but I felt it was a complex and an important enough topic to cover in a blog.

The NYT piece posits that a hacker known by his alias “Profexer” was responsible for writing the P.A.S. tool and is now a witness for the FBI after coming forward to Ukrainian police. The P.A.S. tool, the article puts forward, was leveraged by Russia’s intelligence services without his knowledge (not sure how he can be a “witness” then but I digress). The authors of the article previously explicitly stated P.A.S. was used in the break-in of the Democratic National Committee  (DNC) but they had to issue a correction to that (to their credit, folks from NYT reached out to me after I critiqued it on Twitter to try to get the story correct after it was published; I asked for the correction as I’m sure others did but in reading the updated article the correction doesn’t actually address the larger issues so I wanted to cover them here in the blog).

 

Figure 1: Correction Related to P.A.S. and the DNC

Where did they get this assertion that P.A.S. was used in the DNC breach? By tying the GRIZZLYSTEPPE report (which does note that P.A.S. has been used by Russian security service members before) to the DNC breach. The GRIZZLYSTEPPE report has nothing to do with the DNC breach though and was a collection of technical indicators the government compiled from multiple agencies all working different Russian related threat groups. The threat group that compromised the DNC was Russian but not all Russian groups broke into the DNC. The GRIZZLYSTEPPE report was also highly criticized for its lack of accuracy and lack of a clear message and purpose. I covered it here on my blog but that was also picked up by numerous journalists and covered elsewhere. In other words, there’s no excuse for not knowing how widely criticized the GRIZZLYSTEPPE report was before citing it as good evidence in a NYT piece. Interestingly, the journalists didn’t even link to the “Enhanced Analysis” version of the GRIZZLYSTEPPE report which was published afterwards (and is actually much better) as a response to the critiques of the first one.

A major issue exists though with the correction to the NYT article. It changes the entire point of the story. If Profexer isn’t actually a “witness” to the case because P.A.S. wasn’t used in DNC then what’s the message the journalists are trying to get across? Someone who wasn’t working with the Russians, developed a tool that the Russians didn’t use in the DNC case, and didn’t have any insight into any of the Russian threat groups or campaigns cannot be a good witness.

Even after the correction though the journalists draw the readers attention to the breach early and often to continue to reinforce that this gives new insight into that case.

Figure 2: Snippet from NYT Article Referencing DNC Breach and Profexer

And again the journalists explicitly state that Profexer is somehow a witness to what occurred and reference him back again to the election hacking.

Figure 3: Snippet from NYT Article Claiming Profexer is a Witness

The article goes on to note how this changes our thoughts on the Russian groups (APT28 / APT29 or COZYBEAR / FANCYBEAR) and how they operate; the journalists state that using publicly available tools or outsourcing tool development to cyber criminals is against the modus operandi (MO) of the Russian security services. I do not know where the journalists get this claim but they do not source it; I disagree with the claim but I’ll note the burden of proof here is on them with regards to showing where they’re claiming the previous MO and I’ll simply state that there have been numerous publications and reports showcasing Russian threat groups including the security services using other groups and people’s tools and exploits. This isn’t new information and it’s fairly common for many threat groups to operate in this way.

The attribution on APT28 and APT29 is some of the most solid attribution the community has ever done. Numerous cybersecurity firms have covered this group including FireEye, CrowdStrike, Kaspersky, TrendMicro, and F-Secure but we’ve also had government attribution before by the German intelligence services on a breach into their government that pre-dates the DNC breach. A cursory look will reveal that organizations have been tracking this Russian threat group for about a decade now. Yet none of the people who’ve actually covered these groups were cited in the NYT article. Instead the journalists chose to cite Jeffrey Carr and his quote is confusing to most readers because he is trying to detract from the attribution where he states: “there is not now and never has been a single piece of technical evidence produced that connects the malware used in the D.N.C. attack to the G.R.U., F.S.B. or any agency of the Russian government.” It’s almost as the journalists just wanted a contrarian view to look balanced but what an odd selection if not just set up their witness to be even more important.

I want to be very clear on my next critique: I actually don’t think Jeffrey Carr is a bad person. I know he ruffles the feathers of a lot of folks in the community (mine included at times) but on the two occasions I’ve met him in person he’s been an absolutely nice person to me and was civil and well articulated. That being said, he is not an expert on attribution, not an expert on these groups, nor has any reason to be cited in conjunction with them. He’s often widely criticized in the community when he tries to do attribution and it’s often painfully full of 101 intelligence analysis failures. The NYT didn’t do him any favors by including him in this article and seriously detracted from the idea that they understood enough about this topic to cover it. Simply stated: “cyber” is not an expertise, if you are covering a niche topic like attribution or a further niche topic like Russian group attribution you need to use folks who have experience in that subject matter.

Please Stop Arguing About Attribution Without Expertise In It

This is a bit of a big request but it’d be very useful if people stop taking a stance on why attribution is difficult or not and whether or not attribution is right or not if they have never had experience in doing attribution. This is important because the journalists in this article seem to want to help bolster the case against the Russian intelligence services yet make it more confusing. At one point they try to set up their witness as some new smoking gun to be added to the case as a push back to people like President Trump.

Figure 4: Snippet from NYT Article Setting Up the Importance of the “Witness”

Attribution is not about having a smoking gun. Attribution is a good example of doing true intelligence analysis; there are no certainties and you only can come to an assessment such as low, moderate, or high confidence. Almost every single piece of data put forward in that assessment can and should have counters to it. Very reasonable counters as well. It’s why when anyone arguing for attribution argues a single piece of evidence they almost always lose the argument or look silly. It’s simply very rarely about one piece of evidence and is instead the analysis over the total data set. The attribution levied towards Russia for meddling in the U.S. elections is solid. The reason President Trump and others don’t want to accept that has nothing to do with the fact that there hasn’t been a witness or a “single piece of technical evidence produced that connects the malware used in the D.N.C. attack to the G.R.U.” it is because they do not want to accept the conclusion or the reality it presents. There’s nothing that’s going to change this. I’m convinced that if President Putin came out and said “yea it was us” we’d have critics coming forward saying how it’s a false flag operation and it’s actually not true.

But what’s the problem with people arguing these points? It detracts from the already solid assessment. It’s similar to when the FBI wanted to release IP addresses and some technical indicators during the Sony hack to talk about how they knew it was North Korea. I critiqued that approach when it happened here. The basis of my argument was that the FBI’s attribution to North Korea was likely correct but their presentation of evidence as proof was highly misleading. Obviously the FBI didn’t just use those technical indicators to do the attribution, so how could anyone be expected to look at those and be convinced?  And rightfully so people came out and argued against those technical indicators noting they could easily be wrong and that adversaries of any origin could have leveraged the IP addresses for their operations. And the critiques were correct. The technical evidence in isolation was not good. The totality of the data set though was very sound and the analysis on top of it though were very sound.

I often think of this like climate change arguments. You can have 100 scientists with a career in climate studies posit forth an assessment and then two people with absolutely no experience argue on the subject. One of the people arguing for the climate scientists’ position could grab out a single data point to argue and now the person arguing against that first person is arguing against an uninformed opinion on a single data point instead of the combined analysis and work of the scientists. The two people arguing both leave understandably feeling like they won the argument: the original assessment by the scientists was likely right but the person arguing against the data point was also probably right about their argument against that data point. The only people who lost in this debate were the scientists who weren’t involved in the argument and who’s research wasn’t properly presented.

Closing Thoughts

I never like to just rant about things, I try to use these opportunities as things to learn from. All of this is actually extremely relevant to my SANS FOR578 – Cyber Threat Intelligence course so a lot of times I write these blog posts and reference them in class. So with that theme in mind here’s the things I want you to extract from this blog as learning moments (to my students, to the journalists, and to whomever else finds it valuable).

  • If you are doing research/writing on niche topics please find people with expertise in that niche topic (Jeffrey Carr is not an expert on attribution)
  • If you are going to posit that the entire public understanding of a nation-state group’s MO has changed because a single piece of evidence you’re likely wrong (do more homework)
  • If you are going to posit that there is a witness that can change the narrative about a case please talk to people familiar on the case (determine if that type of evidence is even important)
  • If you are going to write on a topic that is highly controversial research the previous controversy first (GRIZZLYSTEPPE was entirely unrelated to the DNC case)
  • Attribution is not done with single pieces of evidence or a smoking gun it is done as analysis on complex data sets most of which is not even technical (attribution is hard but doable)
  • The most interesting data for attribution isn’t highly classified but instead just hard work/analysis on complex scenarios (classification standards don’t imply accuracy or relevancy)
  • Just because someone’s code was used by an adversary does not imply the author knows anything about how it was used or by whom (the threat is the human not the malware)
  • Stop using pew pew maps (seriously just stop; it makes you look like an idiot)

 

Intelligence Defined and its Impact on Cyber Threat Intelligence

August 25, 2016

Michael Cloppert wrote a great piece to argue for a new definition of cyber threat intelligence. The blog is extremely well written (I personally love the academic style and citations) and puts forth a good discussion on operations. Sergio Caltagirone published a rebuttal equally valuable where he agreed with Mike that there is accuracy missing from current cyber threat intelligence definitions but noted that Mike focused too much on operations. The purpose of this blog is not to rebut their findings but to add to the conversation. In many aspects I agree with both Mike and Sergio; I would highlight that the forms of intelligence discussed though are very policy focused (sometimes even military focused) and influence how we define cyber threat intelligence. I do not envision that between these three blogs we’ve settled a long standing debate on intelligence but the intent is to add to the discussion and encourage thoughts by others.

In Mike’s piece the definition he presented for the field of cyber threat intelligence is the “union of cyber threat intelligence operations and analysis” each of which he previously defined. Sergio responded by stating “Intelligence doesn’t serve operations, intelligence serves decision-making which in turn drives operations to achieve policy outcomes.” I agree with this understanding of intelligence to meet policy needs and while Sergio intentionally does not intend to cover all aspects of intelligence outside of policy I believe it is important to consider. Mike teased out at one point that “…’intelligence’ more broadly is a bias toward a particular type of intelligence, and they continue to overwhelmingly focus on geopolitical outcomes.” He gives an example of business intelligence as another form of intelligence and accepts that the basis of intelligence is interpreted information with an assessment to advance an interest. This is where he stops though in an effort to stay focused on defining cyber threat intelligence. This is where I would like to begin.

Dr. Michael S. Goodman, a professor of intelligence studies at Kings College in London, wrote a piece for the CIA’s Center for the Study of Intelligence where he discussed the challenges and benefits in studying and teaching intelligence. He specifically noted that “The academic study of intelligence is a new phenomenon” although the field of intelligence itself is very old. More relevantly to this blog post he wrote that “Producing an exact definition of intelligence is a much-debated topic.” In a non-government intelligence focused piece the University of Oregon has a page dedicated to the theories and definitions of intelligence. There, they cite psychologists and educators Howard Gardner, David Perkins, and Robert Sternberg to assign attributes to intelligence and state that it is a combination of the ability to:

  • Learn
  • Pose Problems
  • Solve Problems

These three attributes are core to any definition of intelligence whether it’s business intelligence, emotional intelligence, or military intelligence. Additionally, the distinctly human component of this process, for those of you considering artificial intelligence as you read this, is harder to capture but likely exists in the ability to pose and solve problems. Machines can pose and solve problems to an extent but how they do that sets them apart from humans. More to the point, how each of us pose and solve problems is influenced at some level by bias. That bias is often an influence analysts seek to minimize so that it does not jade how we analyze problems and the answers we derive. However, that bias in how we pose and solve problems is likely the only distinctly human component of intelligence. That is a discussion for a longer future piece though.

Further in the University of Oregon piece, different types of intelligences are listed from Gardner, Perkins, and Sternbeg. A few are listed below:

  • Linguistic
  • Intrapersonal
  • Spatial
  • Practical
  • Experiential
  • Neural
  • Reflective

These different types of intelligence are not all encompassing and focus on the psychological more than classic government intelligence. However, they offer a more robust view into what it means to be able to process and analyze information which is in of itself core to cyber threat intelligence. I gravitate more towards Robert Sternberg’s understanding of intelligence and specifically his view of experiential and componential intelligence. According to his 1988 and 1997 writings on intelligence experiential intelligence is “the ability to deal with novel situations; the ability to effectively automate ways of dealing with novel situations so they are easily handled in the future; the ability to think in novel ways.” His understanding of componential intelligence is “the ability to process information effectively. This includes metacognitive, executive, performance, and knowledge-acquisition components that help to steer cognitive processes.”

I enjoy these two the most because they seem to map the closest to the idea of intelligence generation and intelligence consumption. In the field of cyber threat intelligence we often hear vendors, security researchers, and companies talk about “threat intel” and standing up teams to do intel-y things but without specific guidance. There is a stark difference in generating intelligence and in consuming it. Most companies are looking for threat intelligence consumption teams (those that can map their organization’s requirements and search for what is available to help drive defense) not threat intelligence generation teams (those individuals who analyze adversary information to extract knowledge which may or may not be immediately useful). A good team is usually the mix of both but with a clear understanding of which one is the priority and which effort is the goal at any given time. Sternberg’s experiential intelligence speaks more to threat intelligence generation whereas his componential intelligence addresses the ability to process, or consume, intelligence. The definitions are not as simple as this but it is thought provoking.

In reviewing Mike and Sergio’s excellent blog posts with the addition of a wider view on intelligence both from a classical, psychological, and philosophical aspect there are attributes that emerge. These attributes mean that intelligence:

  • Must be analyzed information
    • To perform analysis is a distinctly human trait likely due to our influence of bias and our efforts to minimize it (i.e. no $Vendor your tool does not create intelligence) meaning that it is always up to our interpretation and others may have other valuable and even competing interpretations
  • Must meet a requirement
    • Requirements can be wide ranging such as policy, military operations, geo-political, business, friendly forces movements and tactics, or self-awareness; the lack of a requirement would result in intelligence not being useful and by that extension be an inhibitor to intelligence (i.e. overloading analysts with indicators of compromise is not intelligence)
  • Must respect various forms
    • There is no one definition of intelligence but each definition must allow for different ways of interpreting, processing, and using the intelligence

To further qualify to be threat intelligence the presented intelligence must be about threats; threats are not only geo-political in nature but also may encompass insiders. However, I disagree with the notion that there is an unwitting insider threat because the definition of threat I subscribe to must have the following three attributes:

  • Opportunity
    • There must be the ability to do harm. In many organizations this means knowing your systems, people, vulnerabilities, etc.
  • Intent
    • There must be an intention to do harm, if it is unintentional the harm is still as impactful but it cannot be properly classified as a threat. Understanding adversary intention is difficult but this is where analysis comes in understanding the threat landscape
  • Capability
    • The adversary must have some capability to do you harm. This may be malware, it may be PowerShell left running in your environment, and it could be non-technical such as the means to influence public perception through leaked documents

Therefore, I use the following definition, heavily inspired by classic definitions, for intelligence: “The process and product resulting from the interpretation of raw data into information that meets a requirement.” The product may be knowledge, it may be a report, it could be tradecraft of an adversary, etc. Further, I use the following definition for cyber threat intelligence “The process and product resulting from the interpretation of raw data into information that meets a requirement as it relates to the adversaries that have the intent, opportunity and capability to do harm.” (Note that in this definition of cyber threat intelligence the adversary is distinctly human. Malware isn’t the threat; the human or organization of humans intending you harm is the threat.) Each definition is concise but open-ended enough to serve multiple purposes beyond military intelligence.

I in no way think that this solves any aspect of this debate. And I do not feel that my definitions actually conflict with what Mike and Sergio have put forward but are instead meant simply as an extension of the topic. Mike and Sergio are both extremely competent individuals that I am privileged to call my friends, peers, and over numerous occasions mentors. However, their blogs inspired me to explore the topic for myself and this blog was simply my way to share my opining on my findings. I hope it has been useful in some manner to your own exploration.

The Problems with Seeking and Avoiding True Attribution to Cyber Attacks

March 4, 2016

Attribution to cyber attacks means different things to different audiences. In some cases analysts only care about grouping multiple intrusions together to identify an adversary group or their campaign. This helps analysts identify and search for patterns. In this case analysts often use made up names such as “Sandworm” just to group activity together. To others, attribution means determining the person, organization, or nation-state behind the successful intrusion or attack; this latter type of attribution I will refer to as true attribution. There are many issues with true attribution that I want to explore here. However, there are also those that have pushed back on analysts exploring motives to an attack that I also want to call attention to. When dealing with attribution analysts should avoid the extremes: using true attribution inappropriately or being too hypersensitive to perform analysis and explore motives. Good analysts know when to seek true attribution and when to avoid it.

To explore these concepts I will look at true attribution at the tactical, operational, and strategic level of threat intelligence. While these levels should not be seen as a static category it will help shape the discussion. Tactical threat intelligence often deals with those folks who do the day-to-day security such as performing incident response and hunting for threats in the environment, operational threat intelligence refers to those personnel who work to identify adversary campaigns and often focus on aspects such as information sharing and working through organization knowledge gaps, and the strategic threat intelligence category I’ll use to refer to those personnel that sit at senior decision making levels whether it be executives or board of directors members at companies or national government officials and policy makers.

True Attribution at the Tactical Threat Intelligence Level

In my opinion, true attribution at the tactical threat intelligence level is only harmful to good security practices. Trying to identify who was responsible for the attack seems like a good idea to help shape security practices. As an example, an analyst who thinks that China is in their network might begin looking for intellectual property theft and try to shortcut their effort to identify the adversary. But think about that for a moment. Because our hypothetical analyst thought China was in the network, they have begun to look at the data in front of them differently. In this case, attribution has led our analyst to the land of cognitive bias. Cognitive biases are especially dangerous when performing analysis as they bias the way you think – and analysis leans so heavily on the human thought processes that it can lead us to inappropriate conclusions. Now, instead of keeping an open mind and searching for the threat in the network our analyst is falling prey to confirmation bias where the analyst is looking at the data differently based on their original hypothesis that China is in the network.

This begs the question though, if the analyst has nothing else to go off of shouldn’t they look for the tactics, techniques, and procedures of China in the network as a starting place? In my opinion that is the role of those often funky sounding made up campaign names or intrusion set names; this is what others sometimes call attribution but not true attribution. An analyst that thinks they know what “China” looks like really only knows previously observed activity. If I tell you to think about what China would be doing in a network you might think intellectual property theft. If I tell you the threat is Russia you might think of cybercrime or military pre-positioning. If I say Iran maybe you think about data destruction. The problem is, that thought process is tied to previously observed activity and it’s also going off of the assumption that previous true attribution you’ve heard is correct. Even if we assume all the previously true attribution was correct though analysts have only ever heard of or seen some of the campaigns by adversaries. Russia has teams that are interested in intellectual property theft just as China has teams that are interested in military pre-positioning. We are biased in how we view nation-state attribution based on campaigns we have seen before and it is difficult to take into consideration what is unknown. The better tactic is in identifying patterns of activity such as “Sandworm” and thinking to previous observed threats tactics, techniques, and procedures as a starting place in how we search the network for threats. Then tactical level threat intelligence analysts aren’t biased by true attribution but can use some element of attribution to learn from threats they’ve observed before while attempting to avoid cognitive biases.

True Attribution at the Operational Threat Intelligence Level

At the operational threat intelligence level the use of attribution needs to fit the audience. Operational level threat intelligence analysts should always attempt to serve as the bridge between the strategic level players and the tactical level analysts. When using the observations and information from the tactical level to translate to strategic level players there can be a role for true attribution, which we will explore later. When translating the observations at the strategic level and operational level to the tactical level though true attribution then again becomes dangerous. The way threat intelligence is positioned should be determined by the audience consuming it.

Consider this: an operational level threat intelligence analyst has been asked to take the campaigns observed in the community and translate that information for the tactical level folks to use. The indicators of compromise and security recommendations that the tactical level personnel should use are independent of attribution. The security recommendations and fixes are based off of the observed threat to the systems and vulnerabilities not the attribution; or said another way if you have to patch a vulnerability you don’t patch it differently if the exploit was Chinese or Russian based.

However, that same operational threat intelligence analyst has been asked to identify the threat landscape, the observed campaigns in the community that are relevant to the organization, and make recommendations for strategic level players that can influence organizational change. Here, the analyst may not be able to prove true attribution based off of observed adversary activity but it is in their best interest to identify patterns and motives to attacks. As an example, if there have been a number of campaigns recently that align with the motives of Chinese actors targeting the analysts’ company the recommendation from the operational level analyst to the strategic members might have them take into consideration how they interact with and do business with China. Here the analyst should use language to structure their assessment that the observed threats are Chinese based such as “high confidence”, “medium confidence”, and “low confidence.” Language such as “it is definitively China” should be avoided. Ultimately analysis is based on incomplete data sets (consider the difference between inductive and deductive reasoning) and the provided information is just an assessment.

At the operational level of threat intelligence analysts should be mindful of their audience and be open to putting forth good analysis based on observed activities, threats, and motives without being definitive on true attribution.

True Attribution at the Strategic Threat Intelligence Level

Strategic level audiences often heavily care about true attribution but not always with good reason. Government leaders and company executives want to know their threat landscape and how it might shape how they conduct business or policy. That is a good thing. However, strategic level players should be careful not to use true attribution when it’s not required.

As an example, if the organization is facing security challenges and is consistently having intellectual property stolen they need to look at the security culture of the organization and the resource investments needed to increase security and minimize risk. This inward look at the culture and security investments should usually be independent of true attribution. The tactical and operational level impacts are going to be the same whether the previous culprits were China, Iran, Russia, or the North Pole. However, if the organization is taking an outward approach to doing business or policy making they may need to consider true attribution. Because true attribution is usually based off of assessments and not usually definitive it should usually be approached as a continuum.

To look at true attribution especially for this level of threat intelligence I highly recommend two resources. First, a paper by Dr. Thomas Rid and (soon to be Dr. – congrats Ben!) Ben Buchanan titled Attributing Cyber Attacks. This paper will get you into the right mindset and understanding of attribution for the second paper I would recommend by Jason Healey titled Beyond Attribution. In Beyond Attribution, Jason Healey discusses the concept of responsibility as it applies to attribution. In short, a nation-state has responsibilities with regards to cyber operations especially if they might have been conducted from within its borders. At one side of the scale, a state can take an approach of prohibiting attacks and actually help other nations when an attack has begun. On the other side of the scale a state actually conducts the attack and integrates their attack with third-party proxies such as private companies for hire or hacktivists.

Analysts should be mindful of this spectrum of state responsibility, as Jason calls it, when considering true attribution and the nature of intelligence assessments. It is difficult to have true attribution and true attribution can be harmful to tactical level security. However, identifying motives in attacks and understanding the spectrum of state responsibility to attacks should be considered at the strategic level so that we are not so hypersensitive on the topic of attribution that every adversary gets to operate without consequence.

Case Study: Cyber Attack on the Ukrainian Power Grid

Let’s take these concepts and apply it to the cyber attack on the Ukrainian power grid. If you’re unfamiliar with the case you can read about it here. In this case, I have been very careful about my wording as I know there are multiple audiences that see my quotes in media or read my reports. On one hand, I teach a threat intelligence course and an ICS/SCADA active defense and incident response course at the SANS Institute. In this capacity most of my audience is tactical and operational level personnel. For those reasons I have often tried to reinforce that attribution in Ukraine doesn’t matter for them. Identifying indicators of compromise to hunt throughout the network, preparing the network to make it more defensible, and applying lessons learned from the Ukraine attack are all independent of true attribution. True attribution simply doesn’t matter for how we apply the lessons learned for security at those levels.

However, I also deal with strategic level players in my role in academia as a PhD student at Kings College London and as a Non-Resident National Cyber Security Fellow at New America where I work with policy makers. For this audience, it is important for me to note that definitive true attribution has not been obtained in the Ukraine attack and may never be obtained. However, in considering Jason’s spectrum of state responsibility we have to look at the attack and realize the potential motives, the larger geo-political setting, and analyze if there are any courses of action strategic level personnel should take. In my opinion, I doubt the Russian government itself carried out the attack. However, the attack on the Ukrainian power grid did not fit any apparent financial motives and the motives aligned with various Russian based actors; whether those are private companies, hacktivists, or senior government officials. Therefore, it is in my opinion and in my analysis that strategic level players should look at the elements of attribution that link to Russian based teams and consider Jason’s spectrum of state responsibility. Even if Russia had nothing to do with the attack there should be an investigation into whether or not it occurred from within their borders. If the attack is state-ignored it sets a dangerous precedent. Senior policy makers in other nations should under no circumstance jump to blaming Russia for anything. However, they should look for international cooperation and potentially an investigation as this is a first-of-its-kind cyber attack on civilian infrastructure that led to a power outage. There is a line between espionage and offense; that line was crossed in Ukraine and we must be careful of the precedent it sets.

Conclusion

In conclusion, true attribution is highly abused in the information security community today. Many organizations want true attribution but do not know how to use it appropriately and many private companies are quick to assign definitive attribution to attacks where they simply do not have the appropriate data to support their conclusions. True attribution makes media headlines and the motives for companies to engage in this activity are significant for that reason. Claims of true attribution do increase international tension; not as significantly as some would assume but they are individual data points to policy makers and national level leaders. However, being hypersensitive about true attribution enforces a culture in this field where nation-states can ignore responsibility such as investigating attacks or policing their borders as is normal in international law and policy in any other domain other than “cyber.” There is a balance to be struck. Knowing how to strike that balance and when to use attribution in the form of group names with no state ties or true attribution in the form of an evolving assessment will help the threat intelligence community move to a more mature point where tactical, operational, and strategic level players can all benefit.

 

*Edit 3/6/2016*

I had a good discussion with some colleagues around this post and wanted to add two points.

  • Richard Bejtlich had a really good blog post on the value of attribution and breaks it down in a number of useful ways. His blog post pre-dates mine but I failed to reference it the first time. It can be found here. I would recommend it as it’s a great read and doesn’t take long to work through.
  • Two peers, Mark and Tim, made a case for tactical level true attribution that I think is actually an interesting one to consider. I would argue that most tactical folks shouldn’t consider true attribution and that it’s highly highly abused and resource intensive with little value in the wider community today. That being said, Mark made the point that in a resource constrained environment it might be a useful factor in prioritization. As an example, if you have a lot of phishing emails or malware samples to look at and you need a place to start, true attribution could be of value as that starting point as long as you try to defeat any biases later on. The reason this could be of value (credit to Mark and Tim on this point) over just attribution of groups is: if you have data that is of use to specific countries (think F-35 fighter aircraft intellectual property being of value to China and Russia more so than Niger) using that information as a starting point and prioritization of your searches could be useful. This also touches on the topic of crown jewel analysis combined with threat intelligence; for anyone interested in that subject check here.  This to me gets closer to the operational level than the tactical level and I would expect operational folks to translate these concepts into a usable form for tactical level analysts instead of expecting them to start this process – but I can see the case for why this would be useful at the tactical level and would agree that it’s an interesting one to consider.
  • (Thanks to the peers that took the time to discuss their thoughts with me. Discussions like these help all of us explore our understanding of a topic and I always find my own learning process enhanced by them).

No, Norse is Not a Bellwether of the Threat Intel Industry but Does Hold Lessons Learned

January 30, 2016

Brian Krebs published an outstanding report today titled “Sources: Security Firm Norse Corp. Imploding” which has led to the emergence of a number of blogs and social media rumblings about what this means for the cyber threat intelligence community. Some have already begun positioning that this is the fall of threat intelligence. I would not only disagree and believe this to be a mostly isolated case but position that if anything this is a good sign of the community’s growing maturity. The purpose of this blog is to discuss why Norse’s potential and impending implosion does hold some lessons learned for the industry but holds no prediction of negative things to come for the threat intelligence community as a whole.

Before elaborating on these points though, I want to start off with the much needed statement about the people at Norse. To anyone in the community that holds strong negative feelings for Norse (and you are not alone) please be conscious that many of the individuals working at Norse were professionals and very talented. Many of the negative feelings towards the company were likely based on the marketing efforts and mislabeling of the content and value of their product; not negativity towards the people that work there. I hope the former employees land softly at their next jobs and I would encourage companies looking to hire to think of these individuals without prejudice.

With regards to Norse it was in many ways a good looking company. It garnered national media level attention through smart placement of their cyber attack map (yes the pew pew cyber map analysts have mostly grown to hate – but it looked good in media). There were some key employees recruited who were well respected in the industry. And it raised tens of millions of dollars in investments to appear as an exciting California security startup. So now that the company is apparently imploding it does seem natural to think that this may be an indication of things to come with regards to the threat intelligence industry and for a ripple effect in investments into this space. However, I would state this as wholly inaccurate although there are some lessons learned here for both investors and security startups.

First, Norse Corp. may have garnered national level attention but most of it was not actually good attention. Also, they billed themselves as a threat intelligence company when, in my opinion, they simply were not. Folks who are familiar with me, or read it in the Krebs report, will remember that I came out very publicly chastising their dangerous assessment that there were Iranian attacks on U.S. industrial control systems. The key reason that they had a bad assessment is actually why Norse was always doomed to fail. The company was interpreting Internet scanning data against their high level sensors as attack intelligence. Most threat intelligence companies rely upon enriched data complemented with access to incident response data of actual intrusions; not scanning activity. Norse also held no verifiable industrial control system expertise but were quick to make assessments about these systems. And further when they stated that there were attacks on control systems by Iran what the data seemed to show was they actually should have said scans against systems trying to mimic industrial control systems by Iranian IP addresses. The effort by them and the think tank AEI to state that there should be policy considerations in the Iranian nuclear negotiations based off of this data is a great representation of what not to do in the industry. Simply put, they were interpreting data as intelligence. There is a huge difference between data, information, and intelligence as I outlined here. While their product and Internet level scanning data was interesting and potentially very valuable for research it was not threat intelligence. So while they may have billed themselves as significant players in the threat intelligence community they were never really accepted by the community, or participating in it, by most leading analysts and companies. Therefore, they aren’t a bellwether of the threat intelligence industry or of other companies having trouble simply because they weren’t really ever in “the industry.” The threat intelligence community can be fairly small and making strategic mistakes can have significant lasting impact. Trust is a huge part of the equation in this community.

Second, this case-study of Norse holds great lessons learned. First, because trust is a significant part of doing intelligence work and in participating in this community there is a requirement for companies to realize they are dependent on the ecosystem and are not living in a bubble. Formal and informal relationships, company partnerships, and information sharing can help companies succeed quickly. It is not a competitive landscape in such that companies should think that success is a finite item where one company’s success means less is available for others. Quite the opposite. As threat intelligence is used more appropriately throughout the industry it will continually open up the market. For example, threat intelligence is meant to make good security programs better or to help give important context and information to strategic level organization decision makers – it is not meant to replace bad security programs or act as a magical solution for security. Second, threat intelligence companies should be very careful in lining up their marketing efforts with an honest assessment of what the company’s product or services actually produce. This should apply to any security startup but it is vital in the threat intelligence community. Whereas claims around general security can be difficult to interpret there are definitive ways to look at company claims in intelligence and dismiss them completely as hype. This dismissal is hard to recover from. Finally, an important lesson learned here is for investors and Venture Capital firms to dig deep not only into what is being shown by the company but also in how they are perceived in the community. There are many “experts” in this community who’ve never held the appropriate positions or roles to ever have been put in a situation to speak with expertise about threat intelligence. As an example, one of my critiques of Norse was that their “intelligence report” on industrial control system attacks was not written by anyone with industrial control system expertise. Just as we would expect a Russian intelligence analyst to have an understanding of Russia or even speak Russian the community and investors should demand that assessments are qualified by actual expertise not just general “cyber” expertise.

Venture Capital firms invest in companies with the expectation of not getting an immediate return on investment. In an overly simplified stereotype most Venture Capital funds expect not to see their returns for five to seven years with events such as an IPO or company merger/acquisition. Following that logic, it is reasonable to believe that investments made five to seven years ago are starting to be looked at for their return on investment to the Venture Capital firms. The landscape for investment will likely become much more competitive. There will be lessons learned from investing in good-sounding but under-performing companies. Investors and industry analysts will demand more proof of claims, understand what hype looks like a bit better, and invest even more intelligently. This is a good thing for the industry. I doubt Norse will be the last company to fail in the threat intelligence industry but the industry and investments into it will likely continue to grow. The focus will be on smarter money.

 

 

Data, Information, and Intelligence: Your Threat Feed is Not Threat Intelligence

July 9, 2015

This was first posted on the SANS Forensics blog here.

 

Threat feeds in the industry are a valuable way to gather information regarding adversaries and their capabilities and infrastructure. Threat feeds are not intelligence though. Unfortunately, one of the reasons many folks become cynical about threat intelligence is because the industry has pushed terminology that is inaccurate and treated threat intelligence as a solution to all problems. In a talk I gave at the DFIR Summit in Austin, Texas I compared most of today’s threat intelligence to Disney characters — because both are magical and made up.

When security personnel understand what threat intelligence is, when they are ready to use it, and how to incorporate it into their security operations it becomes very powerful. Doing all of that requires a serious security maturity in an organization. The biggest issue in the industry currently is the labeling of data and information as intelligence and the discussion of tools producing intelligence.

jp 2-0
Figure 1: Relationship of Data, Information, and Intelligence

One of the commonly referenced works when discussing intelligence is the U.S. Department of Defense’s Joint Publication 2-0: Joint Intelligence. Intelligence has been around a lot longer than the word ‘cyber’ and it’s important to look to these kinds of sources to gather important context and understanding of the world of intelligence. One of the graphics (Figure 1) presented in the publication shows the relationship of data, information, and intelligence. If the cyber threat intelligence community writ large understood this single concept it would drive a much better discussion than what is sometimes pushed through marketing channels.

Every organization has an operational environment. The physical location of the organization, the networked infrastructure they use, the interconnections they have with other networks, and their accessibility to and from the Internet are all portions of their operational environment. This operational environment contains more data than could ever be fully collected. Many organizations have difficulty collecting and retaining packet capture for their environment more than a few days (if at all) let alone all of the data. So collection efforts are often driven by tools that can reach into the operational environment and get data. On limited resources it usually takes analysts understanding where the most critical data is located and to collect it using the best tools available. Tools are required to make the most out of data collection efforts. The data in this form is raw.

This raw data is then processed and exploited into a more usable form. As an example, the packet capture that is run against an intrusion detection system generates information in the form of an alert. There should be more data than information. The information may have a sample of the data, such as the portion of the packet capture that matched the alert, and it is made available to the analyst with some context even if only “this packet matched a signature thought to be malicious”. Information can give you a yes or no answer. Another example would be an antivirus match against malware on a system. The raw data, the malware’s code, is matched against a signature in the antivirus system to generate an alert. This alert is information. It answers the question “is malware present on the system”. The answer could be incorrect, maybe the match was a false positive, but it still answered a yes or no question of interest. Tools are not required to make information but it is very inefficient to create information without tools. Most vendor tools that make claims of producing “threat intelligence” are actually producing threat information. It is extremely valuable and necessary for making the most of analysts’ time — but it is not intelligence.

Various sources of information that are analyzed together to make an assessment produce intelligence. Intelligence will never answer a yes or no question. The nature of doing intelligence analysis means that there will only be an assessment. As an example, if an intelligence analyst takes a satellite photo and notices tanks on the border of Crimea they can generate information that states that the tanks are on the border. It answers a yes or no question. If the intelligence analyst takes this source of information and combines it with other sources of information such as geopolitical information, statements from political leaders, and more they could then make an assessment that they state with low, medium, or high confidence that an invasion of Crimea is about to take place. It is impossible to know the answer for sure — there cannot be a yes or no — but the analysis created an intelligence product that is useful to decision makers. There should also be far more information than intelligence; intelligence is a refined product and process. In the cyber field we would make intelligence assessments of adversaries, their intent, potential attribution, capabilities they may be seeking, or even factors such as their opportunity and probability of attacking a victim. The intelligence can produce useful knowledge such as the tactics, techniques, and procedures of the adversary. The intelligence can even be used for different audiences which usually gets broken into strategic, operational, or tactical level threat intelligence. But it is important to understand that no tool can produce intelligence. Intelligence is only created by analysts. The analysis of various sources of information requires understanding the intelligence needs, analysis of competing hypotheses, and subject matter expertise.

By understanding the difference between data, information, and intelligence security personnel can make informed decisions on what they are actually looking for to help with a problem they face. Do you just want to know if the computer is infected? Threat information is needed. Do you just want raw data related to various threats out there? Threat data is needed. Or do you want a refined product that makes assessments about the threat to satisfy an audience’s defined needs? That requires Threat intelligence. This understanding helps the community identify what tools they should be acquiring and using for those problems. It helps guide collection processes, the types of training needed for security teams, how the security teams are going to respond, and more.

There is no such thing as threat intelligence data, there are no tools that create intelligence, and there is limited value for organizations that do not understand their operational environment to invest in threat intelligence. But when an organization understands the difference between data, information, and intelligence and understands their environment to be able to identify what constitutes a threat to them then threat intelligence is an extremely useful addition to security. I am a big believer in cyber threat intelligence when it is done correctly. It’s why I worked with Mike Cloppert and Chris Sperry to co-author SANS578 — Cyber Threat Intelligence. It is unlikely though that your threat feed is really threat intelligence. But it may be exactly what you’re looking for; know the difference so that you can save your organization time and money while contributing to security.