Series entries: First part, Second part, Third part, Fourth part, Fifth part
Note 1: This series of posts is a narration of a forensic analysis of a totally fictional incident response case study (but told, we hope, in a didactic and humorous way). If you want a version with the same technical dose but with less narrative, you can check the video of the talk that the author gave at the XIV CCN-CERT STIC Conference, or take a look at the slides of the presentation.
Note 2: These posts reveal a forensic analysis workshop encompassed within the incident response. There will be some things that could be done more efficiently and elegantly, but the idea was to make them simple so that they are easy to understand. And like any practical workshop, you can take advantage of it in several ways: you can download from LORETO the evidence already worked to follow the case step by step, you can download the raw evidence to do your own investigation… or you can play the CTF DFIR that we have prepared and that will be unfolding the case as you respond to the various challenges.
There are few good months for cybersecurity in a Public Administration, and November is not one of them: projects must be closed, status reports must be made, and it is necessary to make sure that the entire budget is executed (and justified with the corresponding invoices).
Angela de la Guarda, CISO of the MINAF (Ministerio of Alegría y Felicidad, Joy and Happiness in English) has not had a particularly happy year: the pandemic forced the deployment of teleworking for a large part of the personnel, which caused a significant overload on all ICT personnel… and even more so on her and her team, responsible for ensuring the security of all systems.
To make things worse, in the usual restructuring with each change of government, the MINAF has been entrusted with the mission of “unifying all state agencies with skills on joy and happiness.” This has led to the absorption of several entities of different sizes, among which the FFP (Federation of Patron Festivities, in charge of managing all town festivals in Spain) stands out. The FFP has, so to speak… a rather relaxed view of cybersecurity, which made assimilation highly conflictive. In the end, the upper echelons have ruled, and the MINAF finally absorbed the FFP.
It’s 5:00 p.m., and Angela is checking emails to finish the first of a week full of reports and go home, when Salvador Bendito (MINAF security analyst) calls her on her mobile: “Boss, we have a very serious problem. One of the canaries has jumped: we are losing the goalkeepers. All of them”.
Chills run down her spine: Angela and Salvador have designed a network of honeytokens and special alerts that they call “canaries” (like those used in the past in mines to warn of gas leaks), which should only go off when the MINAF is under attack (that is, a very low false positive rate). And the “goalkeepers” is what they call the antivirus of the computers (because in many cases they end up being the last line of defense against an attack).
The “canary” is designed so that when a computer’s antivirus (for whatever reason) is disabled, it will generate an alert. And as the CIA spies used to say with their “Moscow Rules”: once is an accident, twice is a coincidence and three times is an enemy action. In this case, 1,200 alerts indicate that we are facing something very serious: a possible ransomware attack. Many ransomware families have executables that are well recognized by antivirus, so a common tactic is to disable them to smooth its path. The fact that this is happening simultaneously has a direct explanation: the attackers have access to one of the MINAF domain controllers, and have disabled it via a GPO (Group Policy Object).
In the face of an attack of these characteristics, quick reaction is essential. Fortunately, Angela has an incident response plan that addresses exactly this scenario, so she can immediately (and with authorization already granted) carry out the following actions:
- Communicate the incident to the instant messaging group used by all MINAF heads of service and ICT deputy directors.
- Cut the Internet connection (to avoid possible exfiltration of information or actions by attackers).
- Instruct Operations to launch a snapshot of all critical servers (since there are still no ransomware symptoms, it will facilitate recovery).
- Verify that “Lucky” (the domain controller that is only turned on once a week to synchronize, so that if a ransomware attack occurs there will be a quick recovery) is turned off.
- Turn off all backup machines (to prevent them from being taken down by ransomware).
At 5:15 p.m. there is no sign of encrypted files on any of the computers, so Angela can define some quick, short-term targets within the incident:
- Determine the scope of the incident.
- Contain the attackers quickly and forcefully.
- Find the entry vector used by the attackers.
In a few minutes Salvador returns an interesting fact: according to what you can see in GLORIA (the CCN-CERT SIEM tool), a couple of hours before the massive alert the antivirus was disabled on three MINAF computers, one of them the primary domain controller. These three computers deserve an in-depth inspection, starting with the domain controller (Angela was going to analyze in the first instance the 5 that MINAF has, but this evidence allows her to fine-tune her shot). The evidence acquisition procedure is already well worked out:
- DC VM snapshot, suspend to dump RAM and run CyLR to obtain triage data.
- Memory dump with winpmem and obtaining triage data via CyLR.
The first computer to be analyzed is the domain controller. Angela lifts the snapshot as an isolated VM, and first checks the GPOs that the DC has available, finding two GPOs that should NOT be there: “Default Launch” and “Cleaner Task“.
The first is the one that looks the worst, so Angela goes for it:
The GPO was created at 4:04 p.m., just over an hour ago!
As part of its actions, it copies the v2.exe and v2c.exe files from a folder on the DC to C:\temp directory on the computers.
And creates a scheduled task to run at 6:15 p.m.!
Right now it’s 5.30pm, so Angela has 45min to prevent the ransomware from running on all MINAF computers, so she has to act immediately. She knows that the GPO refresh time is 30 minutes (that is, how often the computers check for new GPOs). Without a second’s hesitation she calls Milagros Pérez (Systems Manager at MINAF), and asks her to deploy a new GPO that DELETES those files, with the idea that it will be deployed in time for it to act before the ransomware.
Meanwhile, Angela does not stay still: just in case there are clients that have not synchronized GPO, she eliminates the GPO “Default Launch” to prevent its propagation, and connects to the critical servers (the rest of the DC, the file servers and mail), manually deleting malicious files and shutting down computers (why take the risk, especially with file servers where a desktop with mounted network drives can cause a disaster).
It is 6.10pm, and in the MINAF ICT corridor a fly can be heard. Everyone is waiting tensely as the clock ticks the minutes one by one. 6:15 p.m. arrives, the moment of truth. Everything seems to stay the same, and the minutes keep passing … 6:20 p.m. … 6:25 p.m. … 6:30 p.m. … and not a single computer with ransomware symptoms. It seems that Milagros has lived up to her name, because her GPO has saved MINAF in extremis.
Breathing in relief, Angela nevertheless has a lot of work ahead of her: what ransomware has attacked MINAF? How did they get in? How did they gain control of the DC? We will see some of these answers in the following articles…
