SolarWinds attack explained: And why it was so hard to detect
The recent hack of major cybersecurity firm FireEye by nation-state hackers as part of a much broader assault that involved malicious upgrades to a common network monitoring product and affected major government agencies and businesses. The incident illustrates the disruptive effect that software supply chain attacks can have, as well as the unfortunate reality that most companies are woefully unprepared to avoid and detect such threats.
In a long campaign that began in March, a hacker group believed to be connected to the Russian government obtained access to computer systems belonging to multiple US government agencies, including the US Treasury and Commerce. The news prompted the US National Security Council to convene an emergency meeting on Saturday.
Hackers obtained access to SolarWinds’ infrastructure, a company that makes the Orion network and application monitoring tool, and then used that access to build and distribute trojanized updates to the software’s users.
SolarWinds listed 425 Fortune 500 companies, the top ten US telecommunications companies, the top five US accounting firms, all branches of the US military, the Pentagon, and the State Department, as well as hundreds of universities and colleges around the world, on a page on its website that was taken down after the news broke.
Hackers were also able to gain access to the network of US cybersecurity firm FireEye as a result of the SolarWinds tech supply chain attack, which was revealed last week. Despite the fact that FireEye did not identify the perpetrators, the Washington Post claims that it is APT29 or Cozy Bear, Russia’s foreign intelligence service, the SVR’s hacking arm.
“FireEye has observed this phenomenon at various organizations around the world,” the company said in a statement released on Sunday. “Government, consultancy, technology, telecom, and extractive companies in North America, Europe, Asia, and the Middle East have all been victims. There are likely to be more casualties in other countries and verticals. FireEye has contacted all affected organizations that we are aware of.”
The malicious Orion updates
Between March 2020 and June 2020, software builds for Orion versions 2019.4 HF 5 through 2020.2.1 could have included a trojanized portion. However, according to FireEye’s report, each of the attacks necessitated careful preparation and manual intervention on the part of the attackers.
The attackers were able to change the SolarWinds.Orion.Core.BusinessLayer.dll Orion platform plug-in, which is distributed as part of Orion platform updates. The trojanized portion is digitally signed and includes a backdoor that enables the attackers to communicate with third-party servers. This component is known as SUNBURST by FireEye, and it has open-source detection rules available on Github.
“After a two-week idle duration, it retrieves and executes commands known as ‘Jobs,’ which include the ability to move data, execute files, profile the system, reboot the computer, and disable system services,” according to FireEye analysts. “To blend in with legitimate SolarWinds operation, the malware disguises its network traffic as the Orion Improvement Program (OIP) protocol and stores reconnaissance results within legitimate plugin configuration files. Multiple obfuscated blocklists are used by the backdoor to classify forensic and anti-virus tools running as processes, utilities, and drivers.”
The attackers kept their malware footprint small, choosing instead to steal and use credentials to travel across the network and gain legitimate remote access. The backdoor was used to deliver a never-before-seen lightweight malware dropper known as TEARDROP, according to FireEye. This dropper runs completely in memory and leaves no traces on disk. It was possibly used to launch a customized version of the Cobalt Strike BEACON payload, according to researchers. Cobalt Strike is a commercial penetration testing platform and post-exploitation agent that has been embraced and used by hackers and advanced cybercriminal organizations.
Attackers used temporary file substitution methods to remotely execute their tools in order to prevent detection. This suggests that they replaced a legitimate utility on the targeted machine with their malicious one, ran it, and then replaced it with the legitimate one. A similar strategy involved modifying a legitimate task to run a malicious tool and then reverting the task back to its original configuration.
“In a short period of time, criminals will review logs for SMB sessions that indicate access to legitimate directories and follow a delete-create-execute-delete-create pattern,” the FireEye researchers said. “Defenders may also use frequency analysis to detect anomalous task change by tracking existing scheduled tasks for temporary changes. It’s also possible to keep an eye on tasks to see if they’re running new or unknown binaries.”
This is some of the best operational security that FireEye has ever seen from a threat actor, with an emphasis on evading detection and exploiting established confidence relationships. The company’s researchers, on the other hand, assume that persistent defense can detect these attacks, and have identified several detection techniques in their advisory.
Customers can update to Orion Platform version 2020.2.1 HF 1 as soon as possible to ensure they are running a clean version of the product, according to SolarWinds. On Tuesday, the company plans to release a new hotfix 2020.2.1 HF 2 that will replace the compromised portion and add more security features.
The US Department of Homeland Security has also released an emergency directive instructing government agencies to search their networks for the trojanized portion and report back.
No easy solution
Software supply-chain attacks are nothing new, and security experts have long cautioned that they are among the most difficult threats to tackle because they take advantage of trust relationships between vendors and consumers, as well as machine-to-machine communication channels like software update mechanisms, which are implicitly trusted by users.
Researchers discovered in 2012 that the Flame cyberespionage malware’s writers used a cryptographic assault against the MD5 file hashing protocol to render their malware appear to be legitimately signed by Microsoft and distribute it to targets via the Windows Update mechanism. Although the software developer, Microsoft, was not compromised in this attack, the attackers exploited a flaw in the Windows Update file testing process, showing that software update mechanisms can be exploited to great effect.
In 2017, Kaspersky Lab security researchers discovered a software supply-chain attack by the APT group Winnti, which involved breaking into the infrastructure of NetSarang, a company that makes server management software, and distributing trojanized versions of the software that were digitally signed with the company’s legitimate certificate. After breaking into the development infrastructure of Avast subsidiary CCleaner, the same group of attackers distributed trojanized versions of the software to over 2.2 million users. Last year, hackers obtained access to ASUSTeK Computer’s update infrastructure and distributed malicious versions of the ASUS Liv program.
“From a threat modeling perspective, I don’t know of any company that integrates what a supply chain attack would look like in their climate,” says David Kennedy, a former NSA hacker and founder of security consulting firm TrustedSec. “When you consider what happened with SolarWinds, it’s a prime example of how an intruder might pick any target that uses their product, which involves a large number of businesses all over the world, and most organizations will have no way of integrating it into their detection and prevention strategies. This is not a subject that is currently being debated in the security community.”
Although software that is deployed in organizations can undergo security reviews to decide whether its developers follow good security practices such as patching product vulnerabilities that could be exploited, Kennedy argues that organizations do not consider how that software may affect their infrastructure if its upgrade process is compromised. “It’s something we’re all really immature on, and there’s no simple answer for it,” he says. “Companies need software to operate their organizations, they need technology to grow their reach and stay competitive, and the companies that provide this software don’t think of it as a threat model either.”
“When you’re writing software, you always think of a threat model from the outside in, but you don’t really think from the inside out,” he explained. “How can we build our architecture infrastructure to be more robust to these types of attacks?” “That’s an environment that a lot of people need to be looking at.” Will it be possible to avoid many of these attacks by reducing infrastructure in the [product] architecture? For eg, keeping SolarWinds Orion on its own island helps it to communicate properly, but that’s all there is to it. It’s good security practice to make things as difficult as possible for an attacker so that even if they succeed and the code you’re running is compromised, they’ll have a much harder time gaining access to the information they need.”
Companies should consider applying zero-trust networking standards and role-based access controls not only to users, but also to applications and servers, as software users. Not every user or computer on the network should be able to access every application or server on the network, and not every server or application on the network should be able to connect with other servers and applications. Companies should consider what could happen if a product is compromised as a result of a malicious update when deploying new software or technologies into their networks, and try to put safeguards in place to mitigate the effect as much as possible.
The number of software supply-chain attacks is likely to grow in the future, especially as other attackers see how effective and widespread they can be. Following the WannaCry and NotPetya ransomware attacks in 2017, the number of ransomware attacks against organizations skyrocketed, revealing to attackers that corporate networks are not as immune to such attacks as they thought. Since then, several cybercrime organizations have developed sophisticated strategies that placed them on par with state-sponsored cyberespionage.
Ransomware gangs have also understood the value of exploiting the supply chain and have started hacking into managed services providers to exploit their access into their customers’ networks. NotPetya itself had a supply chain component because the ransomware worm was initially launched through the backdoored software update servers of an accounting software called M.E.Doc that is popular in Eastern Europe.
“Yeah, this is a very good effort,” both organized crime and other nation-state groups are thinking right now, according to Kennedy. They could have encrypted a huge percentage of the world’s infrastructure and made off with enough money to never have to operate again if they reached all the organizations that had SolarWinds Orion built at the same time.
