Email records show that the University of Wisconsin’s cybersecurity system staff rushed to determine if any of its 26 campuses or central offices had been affected by the SolarWinds global hacking incident discovered in December 2020. According to documents, some UW institutions were running the compromised software, although it is not clear whether the attackers stole information or disrupted university networks.
On December 13, 2020, the US Cybersecurity & Infrastructure Security Agency issued an alert stating that computer network monitoring products manufactured by Texas-based SolarWinds were “exploited by malicious actors” and “posed an unacceptable risk to US government agencies.”
On December 16, the international news service Reuters reported that the hackers had been working for the Russian government and had been monitoring internal email traffic for the US Treasury and Commerce departments for months.
History indicates that the attackers did this by hijacking an automated update process used by SolarWinds that was sent to tens of thousands of customers. These included federal, state, local and tribal governments as well as businesses and universities.
On the same day of the Reuters report, UW System Associate Vice President of Information Security Katherine Mayer emailed administrative staff with the subject line “Information Security Incident – Winds solar, ”according to documents obtained by WPR via a public registration application.
“This incident is believed to be likely the result of a highly sophisticated, targeted and manual supply chain attack by an outside nation-state,” Mayer wrote. “This campaign may have started as early as the spring of 2020 and is currently underway. The most dangerous effects of this campaign are lateral movements within infiltrated networks and the exfiltration of data from compromised networks.”
System IT team investigates SolarWinds vulnerabilities
In the following days, Mayer and other IT managers within the UW system worked to identify the computer servers used by 26 campuses, the central administrative office, UW shared services, and the extended campus may have used SolarWinds products.
On December 18, UW Information Security Governance, Risk and Compliance Director Nicholas Davis emailed Mayer asking if central office purchasing staff could review recent purchases of SolarWinds products on campus.
“I’m just not comfortable hearing ‘We don’t have it’ without proof of that, or even what method they used to come to that decision,” Davis said.
Procurement records showed that UW-Eau Claire, UW-Green Bay, UW-Oshkosh, UW-Stout and UW-Stevens Point had made purchases from SolarWinds last year. But that didn’t tell the whole story as the company offers multiple products and only its Orion software was considered compromised.
The initial alert from the US Agency for Cybersecurity and Infrastructure Security included a list of instructions for Orion users. They were told to disconnect any computer system running the software from the Internet, scan memory and operating systems for a list of file names, and analyze network traffic records. A suite alert said SolarWinds attackers were observed to bypass the DUO multi-factor authentication program to access Microsoft Outlook emails.
On December 21, emails to chancellors, campus information security officials and the UW board of directors were distributed within the UW headquarters office.
Campus officials were given instructions on how to check their Microsoft email systems for signs of suspicious activity, while the Regents email provided a summary of the SolarWinds hack.
This email indicated that three UW institutions were using the affected version of SolarWinds but found no malicious code. Further steps were taken to take the servers offline while IT staff downloaded a security patch provided by SolarWinds.
Much of the December 21 email to the Regents referred to two of the 15 UW institutions and included several lines of text and six bullet points that were redacted, aside from a mention that the “estimated number of ‘Affected individuals / cases exposed’ was undetermined. and that an investigation was underway.
System president says institution “could be an easy target”
In an interview with WPR, interim UW System president Tommy Thompson said he couldn’t say if the two institutions mentioned in the email were campuses or how the system was impacted. But he said hackers around the world were constantly trying to break into UW computer systems, and the SolarWinds incident was no different. Thompson compared the current threat to a cybersecurity war.
“A lot of the hackers are actually employed by foreign governments and it’s like a full-time job for them, where they come to work in the morning and are given a computer to use in hacking across America.” Thompson said. .
Thompson said part of the challenge in protecting the UW system from being hacked is its large digital footprint.
“And when you have 26 campuses and 13 universities and thousands of servers, you can well imagine we could be an easy target,” Thompson said.
To reduce the number of ways a potential hacker can infiltrate the UW system’s computer networks, Thompson wants to consolidate them and move the data to a cloud-based system in an effort he calls “IT as service ”.
“And it’s a big job consolidating all the servers and centralizing the computing, “said Thompson.” So it reduces the extent of what can be hacked and how hackers can get into our system. “
Other Thompson initiatives include centralizing academic purchasing and administrative functions through the Procurement-to-Pay Automation Initiative and the Administrative Transformation Program.
Thompson said UW has made good progress and hopes to take “IT as a Service” far enough that the next UW System president can easily finish it.
“But it’s going to get expensive,” Thompson said. “And the resources to do it are such that I’m not sure the legislature will be happy with us.”
Universities appeared to be “collateral damage”
Von Welch is associate vice president for information security at Indiana University and executive director of a collective of academic IT professionals called OmniSOC.
He said the reach of the SolarWinds attack was huge, but the attackers appeared to be focusing on federal agencies. Welch said other clients, like universities, appeared to be “collateral damage” instead.
Information security personnel normally monitor hackers trying to break in from the outside, Welch said, but this time they had already been inside for months by the time the alert was raised. .
“To be honest with you,” Welch said, “I don’t know of any organization that’s sophisticated enough to review their major updates so carefully to catch something like that. It’s incredibly difficult. So once you’ve got you installed that, now all of a sudden you’ve basically got your attacker with access to you inside. And then, you know, you’re playing catch-up. “
Welch said the attack reinforced the idea that any time new software or updates are introduced into a secure computer network, there is a possibility that malicious code may get involved.
“So that just emphasized to us the importance of segmenting our networks, trying to keep the different parts of our systems isolated from each other,” Welch said. “So if any part of our infrastructure is compromised by supply chain attacks like this or a phishing scheme or whatever, that’s not a huge problem.”
Scott White is the director of the cybersecurity program at George Washington University in Washington, DC He told WPR that the The U.S. Agency for Cyber and Infrastructure Security has provided a tool for SolarWinds customers to check for signs that attackers have compromised their networks, and it appears to have worked fine. But he said IT experts still find other vulnerabilities related to the attack.
“There has been other malware that has been transmitted, they believe, through that same backdoor,” White said. “So that’s the problem, isn’t it?” It’s not just the initial attack. That backdoor was open and what other malware was being distributed? ”
White said he expected future SolarWinds hack-related disclosures “for quite a while.”
Editor’s Note: Wisconsin Public Radio is a service of the University of Wisconsin-Madison and the Wisconsin Educational Communications Board.