It’s been awhile since I’ve been able to post to my blog (as it turns out doing a Series A raise for my company Dragos has been time consuming so I apologize for the absence in writing). But it is fitting that my first blog post in awhile has something to do with the GRIZZLYSTEPPE report. I almost got sucked back into writing when I saw the Defense Intelligence Agency (DIA) tweet out the Norse cyber attack map.
Cyber attacks going on right now #DoDIIS17 pic.twitter.com/DhDIXlFygI
— DIA (@DefenseIntel) August 14, 2017
Matt jumped on it pretty quickly though which was great.
A) This is the Norse map
B) Shot angled to cut off the logo
C) Norse map is famously BS
D) What are you doing DIA?https://t.co/sGVPORj6L2— Pwn All The Things (@pwnallthethings) August 14, 2017
I tried to attempt to fill the person in running the account just in case they didn’t understand why folks were less than excited about their presentation.
Just in case you can’t figure out why people will be making fun of you today about the Norse map read this: https://t.co/mFVYpnXOmi
— Robert M. Lee (@RobertMLee) August 14, 2017
But in their responses to me it seemed they didn’t fully understand. They articulated that they use unclassified data for the conference but use classified data at work. Of course the problem wasn’t the data (even though it’s not just unclassified but completely bad/fake data) it’s the idea that a cyber attack map aka “pew pew map” is not a good way to communicate to any audience as its simply a marketing ploy. However, it’s not worth a full blog post so I’ll just instead request everyone to do their homework (should only be a quick Google search) on why pew pew maps are stupid and everyone serious in the discussion should stop using them.
On To the Main Discussion
But on to the main topic. What does Russian election meddling, the GRIZZLYSTEPPE report, and bananas all have in common? Absolutely nothing. Each are individually completely unrelated to each other and people should stop putting any of them together as it ultimately just makes people look silly (to be fair no one’s associated bananas with the election interference yet but it might be a better correlation than the GRIZZLYSTEPPE report).
This discussion was all spawned by an article that the New York Times released on August 16th, 2017 titled “In Ukraine, a Malware Expert Who Could Blow the Whistle on Russian Hacking“. Spoiler alert: he can’t. I went on a bit of a Twitter rant to explain why the article wasn’t good, it can be found here, but I felt it was a complex and an important enough topic to cover in a blog.
The NYT piece posits that a hacker known by his alias “Profexer” was responsible for writing the P.A.S. tool and is now a witness for the FBI after coming forward to Ukrainian police. The P.A.S. tool, the article puts forward, was leveraged by Russia’s intelligence services without his knowledge (not sure how he can be a “witness” then but I digress). The authors of the article previously explicitly stated P.A.S. was used in the break-in of the Democratic National Committee (DNC) but they had to issue a correction to that (to their credit, folks from NYT reached out to me after I critiqued it on Twitter to try to get the story correct after it was published; I asked for the correction as I’m sure others did but in reading the updated article the correction doesn’t actually address the larger issues so I wanted to cover them here in the blog).
Figure 1: Correction Related to P.A.S. and the DNC
Where did they get this assertion that P.A.S. was used in the DNC breach? By tying the GRIZZLYSTEPPE report (which does note that P.A.S. has been used by Russian security service members before) to the DNC breach. The GRIZZLYSTEPPE report has nothing to do with the DNC breach though and was a collection of technical indicators the government compiled from multiple agencies all working different Russian related threat groups. The threat group that compromised the DNC was Russian but not all Russian groups broke into the DNC. The GRIZZLYSTEPPE report was also highly criticized for its lack of accuracy and lack of a clear message and purpose. I covered it here on my blog but that was also picked up by numerous journalists and covered elsewhere. In other words, there’s no excuse for not knowing how widely criticized the GRIZZLYSTEPPE report was before citing it as good evidence in a NYT piece. Interestingly, the journalists didn’t even link to the “Enhanced Analysis” version of the GRIZZLYSTEPPE report which was published afterwards (and is actually much better) as a response to the critiques of the first one.
A major issue exists though with the correction to the NYT article. It changes the entire point of the story. If Profexer isn’t actually a “witness” to the case because P.A.S. wasn’t used in DNC then what’s the message the journalists are trying to get across? Someone who wasn’t working with the Russians, developed a tool that the Russians didn’t use in the DNC case, and didn’t have any insight into any of the Russian threat groups or campaigns cannot be a good witness.
Even after the correction though the journalists draw the readers attention to the breach early and often to continue to reinforce that this gives new insight into that case.
Figure 2: Snippet from NYT Article Referencing DNC Breach and Profexer
And again the journalists explicitly state that Profexer is somehow a witness to what occurred and reference him back again to the election hacking.
Figure 3: Snippet from NYT Article Claiming Profexer is a Witness
The article goes on to note how this changes our thoughts on the Russian groups (APT28 / APT29 or COZYBEAR / FANCYBEAR) and how they operate; the journalists state that using publicly available tools or outsourcing tool development to cyber criminals is against the modus operandi (MO) of the Russian security services. I do not know where the journalists get this claim but they do not source it; I disagree with the claim but I’ll note the burden of proof here is on them with regards to showing where they’re claiming the previous MO and I’ll simply state that there have been numerous publications and reports showcasing Russian threat groups including the security services using other groups and people’s tools and exploits. This isn’t new information and it’s fairly common for many threat groups to operate in this way.
The attribution on APT28 and APT29 is some of the most solid attribution the community has ever done. Numerous cybersecurity firms have covered this group including FireEye, CrowdStrike, Kaspersky, TrendMicro, and F-Secure but we’ve also had government attribution before by the German intelligence services on a breach into their government that pre-dates the DNC breach. A cursory look will reveal that organizations have been tracking this Russian threat group for about a decade now. Yet none of the people who’ve actually covered these groups were cited in the NYT article. Instead the journalists chose to cite Jeffrey Carr and his quote is confusing to most readers because he is trying to detract from the attribution where he states: “there is not now and never has been a single piece of technical evidence produced that connects the malware used in the D.N.C. attack to the G.R.U., F.S.B. or any agency of the Russian government.” It’s almost as the journalists just wanted a contrarian view to look balanced but what an odd selection if not just set up their witness to be even more important.
I want to be very clear on my next critique: I actually don’t think Jeffrey Carr is a bad person. I know he ruffles the feathers of a lot of folks in the community (mine included at times) but on the two occasions I’ve met him in person he’s been an absolutely nice person to me and was civil and well articulated. That being said, he is not an expert on attribution, not an expert on these groups, nor has any reason to be cited in conjunction with them. He’s often widely criticized in the community when he tries to do attribution and it’s often painfully full of 101 intelligence analysis failures. The NYT didn’t do him any favors by including him in this article and seriously detracted from the idea that they understood enough about this topic to cover it. Simply stated: “cyber” is not an expertise, if you are covering a niche topic like attribution or a further niche topic like Russian group attribution you need to use folks who have experience in that subject matter.
Please Stop Arguing About Attribution Without Expertise In It
This is a bit of a big request but it’d be very useful if people stop taking a stance on why attribution is difficult or not and whether or not attribution is right or not if they have never had experience in doing attribution. This is important because the journalists in this article seem to want to help bolster the case against the Russian intelligence services yet make it more confusing. At one point they try to set up their witness as some new smoking gun to be added to the case as a push back to people like President Trump.
Figure 4: Snippet from NYT Article Setting Up the Importance of the “Witness”
Attribution is not about having a smoking gun. Attribution is a good example of doing true intelligence analysis; there are no certainties and you only can come to an assessment such as low, moderate, or high confidence. Almost every single piece of data put forward in that assessment can and should have counters to it. Very reasonable counters as well. It’s why when anyone arguing for attribution argues a single piece of evidence they almost always lose the argument or look silly. It’s simply very rarely about one piece of evidence and is instead the analysis over the total data set. The attribution levied towards Russia for meddling in the U.S. elections is solid. The reason President Trump and others don’t want to accept that has nothing to do with the fact that there hasn’t been a witness or a “single piece of technical evidence produced that connects the malware used in the D.N.C. attack to the G.R.U.” it is because they do not want to accept the conclusion or the reality it presents. There’s nothing that’s going to change this. I’m convinced that if President Putin came out and said “yea it was us” we’d have critics coming forward saying how it’s a false flag operation and it’s actually not true.
But what’s the problem with people arguing these points? It detracts from the already solid assessment. It’s similar to when the FBI wanted to release IP addresses and some technical indicators during the Sony hack to talk about how they knew it was North Korea. I critiqued that approach when it happened here. The basis of my argument was that the FBI’s attribution to North Korea was likely correct but their presentation of evidence as proof was highly misleading. Obviously the FBI didn’t just use those technical indicators to do the attribution, so how could anyone be expected to look at those and be convinced? And rightfully so people came out and argued against those technical indicators noting they could easily be wrong and that adversaries of any origin could have leveraged the IP addresses for their operations. And the critiques were correct. The technical evidence in isolation was not good. The totality of the data set though was very sound and the analysis on top of it though were very sound.
I often think of this like climate change arguments. You can have 100 scientists with a career in climate studies posit forth an assessment and then two people with absolutely no experience argue on the subject. One of the people arguing for the climate scientists’ position could grab out a single data point to argue and now the person arguing against that first person is arguing against an uninformed opinion on a single data point instead of the combined analysis and work of the scientists. The two people arguing both leave understandably feeling like they won the argument: the original assessment by the scientists was likely right but the person arguing against the data point was also probably right about their argument against that data point. The only people who lost in this debate were the scientists who weren’t involved in the argument and who’s research wasn’t properly presented.
Closing Thoughts
I never like to just rant about things, I try to use these opportunities as things to learn from. All of this is actually extremely relevant to my SANS FOR578 – Cyber Threat Intelligence course so a lot of times I write these blog posts and reference them in class. So with that theme in mind here’s the things I want you to extract from this blog as learning moments (to my students, to the journalists, and to whomever else finds it valuable).
- If you are doing research/writing on niche topics please find people with expertise in that niche topic (Jeffrey Carr is not an expert on attribution)
- If you are going to posit that the entire public understanding of a nation-state group’s MO has changed because a single piece of evidence you’re likely wrong (do more homework)
- If you are going to posit that there is a witness that can change the narrative about a case please talk to people familiar on the case (determine if that type of evidence is even important)
- If you are going to write on a topic that is highly controversial research the previous controversy first (GRIZZLYSTEPPE was entirely unrelated to the DNC case)
- Attribution is not done with single pieces of evidence or a smoking gun it is done as analysis on complex data sets most of which is not even technical (attribution is hard but doable)
- The most interesting data for attribution isn’t highly classified but instead just hard work/analysis on complex scenarios (classification standards don’t imply accuracy or relevancy)
- Just because someone’s code was used by an adversary does not imply the author knows anything about how it was used or by whom (the threat is the human not the malware)
- Stop using pew pew maps (seriously just stop; it makes you look like an idiot)