Do you remember the first post when we talked about what is and what is not Threat Hunting? Well, an essential part of it is the generation of intelligence.
It’s good that we are the best at detecting abnormal behavior, but if all that acquired intelligence is not transformed into structured and repeatable information we lose one of the most valuable parts of the process.
Structured, so that anyone other than the author can use it and understand it. Repeatable in the better way possible, so that the detection teams can generate alerts with it or so that any other analyst can perform the queries in the most comfortable way possible.
In our laboratory we are going to use another part of HELK, the all powerful Jupyter Notebook.
What is Jupyter Notebooks?
To begin with, let’s imagine a notebook, in which we can write our advances, arguments and conclusions.
On the other hand, let’s imagine a program, a script, something functional that is able to process data, to connect to our laboratory… all the wonderful power of programming.
All that together is a Jupyter Notebook.
Whoever is already working with the project will be pulling the hair out of that description, but you’ll agree that the simplified idea is that ;-)
A more realistic explanation would be that Jupyter Notebooks is the evolution of the IPyhton Notebooks project, a project that was created with the purpose of creating documents capable of processing data and showing an aesthetic similar to a regular text document.
Jupyter works as a web service, which will process our files with “.ipynb” extension and will allow us to create, model, process and share our intelligence. This technology is applicable to any field imaginable, being very known in data processing.
We are going to forget about all the technology behind, since HELK makes the work much easier.
Our first notebook
Let’s see how to access the console. When we installed HELK, at the end of the installation the addresses of all the services were shown, among which was Jupyter’s web console. Among other thingss:
HELK JUPYTER SERVER URL: https://192.168.129.137/jupyter/ HELK JUPYTER
CURRENT TOKEN: 8dd6ade6964ca5600b9250d7d92726d36e769192fca793b5
Each installation will have a different IP and token.
The IP is the same as for the rest of the lab. If you don’t have the token, that’s okay, you have to execute this command in the virtual machine to get it back.
sudo docker exec -ti helk-jupyter jupyter notebook list | grep "token" | sed 's/.*token=\([^ ]*\).*/\1/'
By accessing Jupyter’s URL and entering the token, we enter Jupyter’s home folder.
As you can see, we have 4 folders.
- In the “datasets” folder, we save the events we work with.
- In the “demos” folder there are preloaded examples of the HELK project.
- In the “sigma” folder, the creator of HELK has done a titanic job to transform a total of 378 sigma rules into queries that can be made from these notebooks.
- In the “tutorials” folder there are tutorials that, besides teaching us how Jupyter Notebooks work, allow us to learn about Python, Numpy, Pandas…
It is highly recommended to spend some time on the tutorials. There is one tutorial to learn Python very 101, and besides learning Python it shows the incredible potential of Jupyter Notebooks.
Although it is not my intention to make an article detailing the use of notebooks, we will make a test one and if there is interest (say it in the comments) I will write an entry entering more in detail.
Creation of Jupyter Notebooks
Located in the “demos” folder, on the right side we see the “New” button, which gives the option to select the Kernel that will interpret our notebook. In this case we select “Python 3“.
We already have our first notebook! Now you just have to do what we want.
In the second post we used one of the Mordor datasets, “empire_wmic_add_user“. Now we are going to automate the search for this technique.
In the first box we add a detailed title and description, so that anyone can understand our notes. Important to add the ATT&CK tactic in the title.
Jupyter plays Markdown, so notebooks can be decorated as much as you want. Remember to check the “Markdown” selection in the toolbar.
In the next cell we make the imports that we will need.
from elasticsearch import Elasticsearch
from elasticsearch_dsl import Search
import pandas as pd
from IPython.display import display, HTML
In another cell we created an Elastic Python client.
es = Elasticsearch(['http://helk-elasticsearch:9200'])
searchContext = Search(using=es, index='logs-*', doc_type='doc')
And with our client, the consultation is made to Elastic.
s = searchContext.query('query_string', query='(event_id:"1" AND process_command_line:*wmic.exe* AND process_command_line:"process call create *net user *add")')
response = s.execute()
if response.success():
df = pd.DataFrame((d.to_dict() for d in s.scan()))
Note that in the query I have entered the pattern of the command used by Empire. Although this technique could be used by other software or malware, it will always involve an attempt at persistence to create a user.
Now we have our data stored in a Panda DataFrame, and all that is left is to display it.
display(HTML(df[['host','process_name','process_parent_name','process_command_line']].to_html()))
Once the notebook is created, we save it with a name. In my case “[ T1136] Empire WMIC create user“.
When we leave and come back in, we will see our formatted text. I show mine:
Now, if we position ourselves in the first cell and press “Run” the commands will be executed. It’s okay to execute the ones with text.
As we go, each piece of code will be executed. When we reach the end, we will see something like this:
Bingo! We have caught the event that executed the persistence. Now, every time we want to perform that query on our lab, we just need to enter and execute our notebook: we have transformed knowledge into intelligence.
Here I leave a series of articles about the creator of HELK talking in detail about the Jupyter Notebooks, essential to exploit this incredible tool to the fullest.
See you at the next hunt!
