Welcome to Cyber Security Today. This is the Week in Review edition for the week ending December 17th. I’m Howard Solomon, contributing writer on cybersecurity for ITWorldCanada.com.
In a few minutes I’ll be joined by Terry Cutler, head of Montreal’s Cyology Labs, to discuss some of the bigger stories of the past week. But first a roundup of the headlines:
The biggest story of the week — and perhaps of the year — is the race to fix IT systems before threat actors exploit the log4j2 vulnerability. Terry’s been helping organizations on this so we’ll have a lot to talk about.
We’ll also look at the lessons learned from a ransomware attack that crippled the healthcare system across Ireland.
A workforce management solutions company called UKG is still dealing with the impact of a ransomware attack that started last Saturday. Organizations affected use the Kronos Private Cloud for accessing UKG Workforce Central for keeping track of employee attendance. So the attack may affect the ability of companies to calculate. UKG said it can’t access any customer data. Also affected are hosted versions of TeleStaff and UKG solutions for banks. All of these solutions are still offline. The company said on-premise versions of UKG applications aren’t affected.
The U.S. government’s Department of Homeland Security will pay cybersecurity researchers to find software vulnerabilities in its systems. It’s the latest bug bounty program offered by organizations to help improve security of their websites and applications. However, only invited and vetted researchers can participate. In addition to improving DHS security the goal is to create a model that can be used by other government agencies.
The Microsoft Teams conferencing solution now has improved security. End-to-end encryption has been added to make it harder for unapproved people to tap into one-to-one conversations. IT administrators will have the power to enable and control the capability.
IT administrators who oversee Microsoft Exchange servers should be alerted to a vulnerability. According to researchers at Kaspersky, the problem is in the IIS web server part. It can be tricked into abusing the Outlook Web Access component to steal usernames and passwords. So far victims have been seen in Asia. However, there’s evidence organizations may have been targeted in Europe, so this tactic may spread. Exchange administrators need to keep their servers patched, watch for suspicious activity and make sure users take advantage of multifactor authentication to protect their passwords.
Finally, the Oregon Anesthesiology Group now says a July cyber attack on its IT system resulted in the theft of data on 750,000 patients and 522 current and former employees. It learned of the theft from the FBI, which got the data after seizing an account belonging to a Ukrainian hacking group. The FBI believes the hackers exploited a vulnerability in the company’s firewall. The medical group says the hackers were able to data-mine the IT administrator’s username and password to access its encrypted data.
(The following is an edited transcript. To hear the full conversation play the podcast)
Howard: I’m going to bring in Terry Cutler. Your most recent issue is helping clients deal with the log4j vulnerability. What is it?
Terry: Log4j is part of the Apache logging service that records user activity and behaviours of [Java-based] applications. Software developers use this so they can log how their application is doing what the users are doing. It’s being used in corporate networks, websites applications.
It’s pretty widespread. The service is being used in everything from Apple to Minecraft. This flaw allows attackers to steal information or steal data from a server or install malware on it and basically take full control.
Howard: As I understand it hackers don’t need to create a phishing email message with a sophisticated attachment or links in order to spread an attack through this vulnerability. So How can an IT system be infected?
Terry: The attacker could just use what’s called remote code execution. It’s basically the attacker can launch a remote code and once they get into the system they can again install malware take full control. But the other issue that we’re seeing is once these attacks have been launched they can show up [on a victim’s computer] six to eight hours later after they’ve been sent so it’s very very hard sometimes to indicate that the attack is happening right now.
Howard: One of the problems is as soon as word gets out about a critical vulnerability threat groups try to exploit it, so there are reports that nation-states as well as criminals are attempting to use this vulnerability to steal data, install ransomware and implant cryptomining apps.
Terry: We’re seeing also that the exploits are being developed so that they can gain a foothold into the environment. They’ll set up what’s called a remote access trojan, or RAT. The newest one that they’ve launched is called Orcus, and it’s basically a back door into the system. What’s interesting is that it creates what’s called a remote bash shell What this means is that if you try to attack a system that’s behind a firewall. The firewall is going to block it but because the system was vulnerable (the RAT) does a reverse callback to the attacker’s machine, which means the firewalls’ going let it go through because if it came from inside [the network] the back door was established. Now attackers can come in and regain control of the server whenever they want.
We’re also seeing attacks against access points. They’re inside email headers. An attacker can paste these links on social media or SMS [texts] or even some support request tickets. The other issue is that because it involves a [software] library malware scanning tools won’t always see it. Luckily the log4j developers put a version [number] inside their banner, and that shows us at least that there’s at least some type of log4j file there so we can find it.
Howard: What have you been doing for organizations that have been asking for help?
Terry: Everything from working with third-party vendors to help them patch their systems, running some vulnerability assessments to try and find the issues we’re looking for, such as IP signatures at the network level. Looking for indicators of compromise is an important piece because some of these attacks can happen at a later time. You want to be listening at the network level to get some visibility in the environment to see what’s leaving the company, if there’s any beaconings happening. And of course we’re working with vendors that have XDR capabilities [endpoint detection and response] to at least correlate to see if there’s anything weird going on.
Here’s one of the issues that we’re dealing with right now: We’re scanning for these flaws in log4j, so if you’re running anything between versions 2.0 to 2.14. you’re vulnerable. You’ve got to get to version 2.15 — but a new patch just came out overnight while the scan we’re doing is going on. You see 2.15 doesn’t stop a denial of service attack, so now you got to go to 2.16. This is a ‘patching in progress.’
Howard: How concerned are the companies that are contacting you for help?
Terry: They’re actually very concerned because right now they have no idea if this software is actually running inside the company. Maybe their software is built In-house, but the developer doesn’t work there anymore. They just don’t Know. So we’ve got to go in there and try and find if these servers are running that [bad] version of software.
Howard: Are enough organizations in your opinion taking this seriously?
Terry: They are. My phone has definitely been blowing up. The problem is there are not enough resources to help all these guys in a timely manner. We’re trying our best to get this done because remember this vulnerability is only 96 hours old, so the cyber criminals have a leg up on this. They know that software that companies can’t just start patching this software overnight.
Howard: You said the vulnerability is a couple of days old, but that’s the public announcement. I’ve seen news stories where Companies like Cisco Systems say they’ve seen evidence that somebody was trying to exploit this as early as December 1.
Terry: The other thing is Java has been around for ages. And there haven’t been many exploits developed for Java, and so now this is fairly new and we are trying to re-learn how the whole Java and log4j works and piece this all together.
Howard: The Us government has told federal departments and agencies they have to patch or mitigate systems by December 24th. Some experts say it may take years for IT departments to root out and fix all the applications where log4j is used.
Terry: I think it will [take years] because nobody has a full grasp on their environment to see where this is installed. It’s not like you can always find it in your installed software list. You can look for JAR files [Java ARchive files, which could indicate the presence of log4j]. There’s also been a list of hashes that have been compiled and you can download them online to see if any of these leakages have happened or if it’s being discovered on the network. But again, luckily, the developers left the versioning code inside the banner of the software. So at least it could be discovered.
Howard: The other thing I’ve seen is that log4j is used in industrial internet-connected applications that control machinery and valves and pipelines and such. These are called supervisory control and data acquisition applications, SCADA systems. What about the effort that it will take to go through all of those systems?
Terry: Same thing, because some of these SCADA systems don’t like to be scanned. So if you try to run and a vulnerability scanner against it or other types of tools it might break the device. It might crash it, and sometimes these SCADA devices are critical to infrastructure. So they may have to be scanned at 2 a.m. The other issue we’re going to have is that some of these devices may not be able to be updated, or they cannot be replaced. We’re pretty much scrambling to find out what we’re going to do.
Howard: What’s your advice for IT professionals and application developers?
Terry: Because we’re still in the discovery phase you need to run these scanners to detect the existence of that software. You want to upgrade to at least Java 8, which has the log4j version 2.16. There is another mitigation component. This is gonna be a little technical but you could set the system property the Java Naming and Directory Interface [JDNI] to “true.”
Howard: I want to turn now to the report on the ransomware attack on Ireland’s health care network earlier this year. This is a network that’s run by what’s called the Health Services Executive, or the HSE. It serves many hospitals and health care services across the country. Some of them are independent, and so it’s not like the IT department has complete control over all the systems that are attached. This attack caused an awful lot of trouble: 80 percent of the health care data in the system was encrypted, hospitals were forced to cancel outpatient treatment. It cost $600 million dollars to scour the systems to make sure all the malware was eradicated. And it all started with one person clicking on a malicious document in an email. There was an extensive report that was released this month that looked into the attack. Terry, what did you think when you went through that report?
Terry: I think I can sum that up pretty simply into one word: Terrifying. Here’s a perfect example of what a disaster will look like. I’ve worked in healthcare [here], and the infrastructure is just so large. There’s a lot of technology there that cannot be upgraded so we’re still dealing with Windows XP, Windows 7 simply because, for example, the software that controls the door mechanisms in the hospitals could only run on Windows XP so they’re forced to keep this legacy software. How do you go to protect against this?
Howard: This report noted that in Ireland there’s a lot of legacy [HSE] systems. The report didn’t say that having all those legacy systems was a major factor in this in this particular attack. In fact, what struck me was that the antivirus system that was on the network was set to only monitor suspicious activity and not to block it. That’s a pretty big mistake there. Perhaps the attack could have been slowed could have been checked.
Terry: Here’s the issue with antivirus if you’re just dealing with the traditional stuff. These are signature-based technologies. It means that every four hours it’s going to go and download the latest signatures of the viruses. But the newest attacks coming through are file list malware. So what it means is you’re going to receive an email. Let’s it could be an Excel file or a Word document. It comes through email and it comes in clean. But the moment you turn on [document] macros [because the message asks you do to so] all of sudden it downloads the payload into your system. And that’s why EDR [endpoint detection and remediation] is so important. EDR looks for weird anomalies and weird behaviours that are happening on a system.
Howard: And one of the things that the report suggests was that the HSE really relied on antivirus for defending the entire network.
Terry: That’s not good anymore. The antivirus technologies just can’t keep up. That’s why you need to have the endpoint detection and response to complement the traditional antivirus technology, or replace it, because today’s threats are just so sophisticated…..
EDR would recognize unusual behaviour and lock it down or alert the headquarters that something’s going on and trigger a critical alert saying watch out for this.
Howard: We spent a lot of time here talking about technology. There’s also the human administration response and oversight of the IT network. That was missing in this particular case. The first victim’s computer hadn’t had its antivirus updated for twelve months.
Terry: It doesn’t surprise me because the environments are so large. When you’re dealing with tens of thousands of computers maybe you’re looking at your [admin] dashboard. You can see this computer is on is offline. But maybe the antivirus just couldn’t update it because maybe it’s being blocked by a firewall or something’s going on and when in fact it’s actually online. So because the antivirus isn’t updated the user checks his emails and next thing you know they get hit.
Howard: And the HSE had an IT staff of 350 people, but only 15 of them were full-time responsible for cybersecurity.
Terry: That doesn’t surprise me because of our shortage in cybersecurity right now. We’re almost 2 million personnel short around the world. They don’t have the proper training to stay up to date, so a lot of times you’re just taking a traditional IT guy and tasking him with cybersecurity. Now he’s going on YouTube trying to find out all the latest tools and tactics and techniques to help defend the organization. So they [management] needs to invest in proper training. Also, look at professional tools that can help automate some of these processes. The biggest thing I could say, especially in health care, is IT has no visibility onto their network. So when these attacks happen they have no idea what’s going on. They’re scrambling.
Howard: One thing that this report did make clear is that there were lots of signs of suspicious activity and the IT staff either missed or ignored them.
Terry: Either because they didn’t know how to deal with it or maybe the [support] ticket that was open wasn’t maybe detailed enough, or they didn’t have the skills or didn’t have the tools.
Howard: I noticed in the report there were references to the attacker’s ability to get into administrator accounts. There was no reference in the report on whether these were protected with multifactor authentication I’m sure if they were that would have greatly helped to check the spread of this attack.
Terry: Definitely. But again if there are exploits happening in the system it’ll bypass all that stuff. It’ll get system-level permissions and then from there they can start digging their way through the network. I think in the future is that you’re going have a special group in every organization, we’ll call it a centre of excellence, where all the cyber guys will be in and will help complement the IT department.
Howard: The report also mentioned that this was a flat IT network, not a segregated network. Tell us how that could have made a difference.
Terry: If they’re segmented it means that if one range of the network gets infected it can’t traverse into other parts of the networks. Segmenting off your network definitely would have helped contain the [HSE] ransomware for sure. And you know, we [here] often see a lot of victims of ransomware after the fact and their networks are all flat too. Which also is a reason why their backups get hit, because backups aren’t protected.
Howard: The report mentions that there weren’t regular backups. HSE only backed up irregularly. One thing that helped HSE was the ransomware gang that was behind this was embarrassed by the public uproar of an attack against the hospital system and they released the decryption keys for the ransomware. That very much helped IT administrators unscramble the encrypted data. Although even still it took several months before the network was fully restored. And that’s a lesson I think for all companies: Don’t expect any such treatment from most ransomware gangs.