Articles in the series “Exchange forensics: The mysterious case of ghost mail”: [1] [2] [3] [4]
(Note: This is a fiction story, the characters and situations are not real, the only real thing is the technical part, which is based on a mixture of work done, experiences of other colleagues and research carried out. If you want a version with the same technical dose but with less narrative, you can consult the video of the talk that the author gave at the 11th STIC Conference of the CCN-CERT here).
Another day in the office, with a list of pending tasks to plan longer than the beard of Richard Stallman and none of them entertaining: reports, documentation of a couple of projects and the preparation of a meeting is what the menu of the day offers for almost the entire week.
Luckily, the saying that “no plan survives contact with the enemy” in this case works in our favor. The phone rings, and my boss goes straight to the point: “A YARA rule has been triggered from the ATD group in CARMEN of [Redacted] (entity whose identity we are going to leave anonymously, calling it “the Organization” from now on). Take your stuff and rush over there.”
The adrenaline rush at the thrill of the hunt is instantaneous: ATD is our internal name of a group of attackers that we hunted a few months ago on another client, and our reversers ripped the malware open from top to bottom without mercy. The analysis allowed us to detect a series of particular “irregularities” in their way of acting, which allowed us to generate a series of high fidelity YARA rules (that is, false positives practically null). If it was triggered on CARMEN (our advanced intrusion detection tool), then 99% sure to be infected”.
A laptop, a 4Tb hard drive for forensic captures, a write blocker, USB flash drives with hot analysis tools and live CD, and a flamethrower… the incident response backpack is ready. A handful of candy from the jar (you know when you start managing the incidents but not when you finish them) and we catch a taxi straight to [Redacted].
María (the security technician of the Organization) has done an excellent job getting more information about the possible attack, and she offers us this preliminary information:
- The YARA alert was triggered in an email attachment, and made reference to an international event that will occur within two weeks (a very common technique in advanced spear-phishing attacks).
- The email was sent to an executive of the Organization, who being in charge of two areas, is very likely to have access to sensitive information.
Lady luck smiles on us again because the manager is traveling abroad and only reads his emails on his mobile (where the chances of infection are much smaller). We reach the manager, we tell him in broad strokes what happened and we obtain his authorization to request Systems to retrieve the mail from his user mailbox (be very careful not to do these things without the appropriate consents or the wrath of the LOPD/RGPD will fall on you, you’ve been warned).
The attachment (a Word document) has nothing new: information about an international business meeting about blah blah blah… and an obviously malicious attachment. YARA’s half second on the attachment confirms the rule, and another five with the oletools allows to identify that the mail contains a malicious macro. The Word is quickly exploited in a Windows 7 virtual machine with dnschef to emulate Internet connectivity, and spits a URL at us out without resisting (according to what we know about this group to this day, the macro acts as a dropper, downloading the real malware from this URL).
We capture the URL, and by falsifying the User-Agent of the request (just in case) we download the executable from the Wi-Fi of the Organization (so it seems that the manager opened the mail, in case the attackers are monitoring the logs of his server).
We compress it with a password, we encrypt it with PGP and we send it to our reverser on call with the subject “ATD – Revise ASAP”. Meanwhile, we continue with containment tasks looking for the URL in the logs (to verify that they have not sent any more emails with malware aimed at that URL). Then we review the logs of the mail gateway, in order to identify the false address where the email came from and especially the server.
This step is very important because it feeds back our threat intelligence operations (attackers usually reuse infrastructure, so it is always good to know it), and above all because they are very good IOCs (Indicators of Compromise). This information can (and should) be shared with other entities (CCN-CERT or INCIBE/CNPIC), to notify other organizations that may be victims of the same attack.
We rummage through the logs of the mail gateway while we ruminate on “with SPF this would not have happened” … and we are surprised not to find it. Strange. We check the mail data in case we have made a mistake on the date, but it is correct. We repeat the search (the greps are sometimes loaded by the devil), and we still cannot find the mail in question.
It’s time to start reviewing other alternatives. We ask Maria to locate the person who supposedly sent the mail (apparently, from the administration area of the manager’s department). She assures categorically that “of course he has not sent it”, but as soon as we read the subject to her and a part of the attachment she recognizes it as his. She affirms with certainty that it corresponds to an event that they have to organize … but that the email is a draft that has not yet been sent to the list of users in which these matters are dealt with.
The conclusion we reach is that it is an internal email of the Organization. This fact clarifies two things: firstly, it explains why it does not appear in the logs of the external email gateway, since it is a purely internal email. And secondly, that our attackers have compromised at least that computer, which indicates that the incident will not be as easy to resolve as it seems…
We return to the phase of the identification of the incident (not without first sharing the IOCs that we have already obtained from the incident to speed up the response in other organizations), and the first thing we do is locate the user’s team that sent the email. Luckily for us (it’s a saying) it is almost 6:00 pm, so we can take advantage of the fact that the user finishes his workday to fall on his computer like hungry fox terriers.
We apply the standard procedure: disconnecting the network, capturing RAM with DumpIt, obtaining triage data with Bambiraptor (the name may sound like a joke, but the tool is great) and turning off the computer by pulling the cable for a dc3dd session of the hard drive via write blocker. For the purposes of the attackers, the user has turned off the computer and has gone home (rule # 7 of the response to incidents: Never let the attackers know that we know they are there).
The forensic copy starts when a colleague walks through the door (in serious incidents, the more people you can put on the ground as soon as possible, the better), so we have a court martial in front of the coffee machine and I update her on the incident. We do the (traditional) rock-paper-scissors game to see how we divide the work, and I’m left with the RAM capture while she begins to dig in the logs of events, Windows and MFT records in search of traces of the attackers.
Volatility, there we go. The first thing we do is throw out our set of YARA rules for ATD on RAM with yarascan, because we should quickly identify the process infected by the malware. Five minutes later it returns with scant results: zero. Oops, our “friends” must have evolved their malware. Time to roll up our sleeves.
The next quick win is usually malfind: we launch it on RAM to detect hidden or injected code in the memory, but it does not return anything interesting either. It’s time to launch all the artillery: pslist, pstree, psscan, cmdscan, netscan, handles, etc … A good time later, we could not find anything malicious in the computer.
I look askance at my partner, who is searching furiously in the log files for some clue, some trace left by the attackers, but is not successful either. If the RAM or triage have not worked, we will have to go for the hard drive.
We can use Autopsy for data recovery, or make a temporary line with Plaso, but first we want to check one thing: we need to see the malicious email sent with our own eyes. We locate the .ost file that stores the user’s mailbox (the Organization has a corporate Exchange synchronized by MAPI with the clients, so instead of .pst we have .ost as storage of the emails).
We open the .ost file with the Kernel OST Viewer utility and search for malicious mail… again without finding it. Well, it may have been deleted. In the end an .ost file ends up being a kind of database, and if it has not been compacted, we can retrieve information with tools like the OST Recovery Tool. We launch the tool on the extracted .ost and the mail still does not appear.
“On equal terms, the simplest explanation is usually the most likely” says the principle of Ockham’s razor. The email does not seem to have left this computer (we leave Autopsy running to look for other files deleted just in case), so the next option is to check if it has been sent by other means.
María confirms that the Organization has an OWA (Outlook Web App, a corporate webmail), as well as an email access for corporate mobiles via ActiveSync. However, the affected user is of an administrative level and does not have access to any of the two resources (the OWA is behind a VPN for which he does not have credentials, nor does he have a corporate phone).
The research leaves us with only one way to investigate: the mail server itself, a Microsoft Exchange 2010. The analysis, what we find (and what not), we will see in the following article…
