TLS client fingerprinting with Bro

2 de February de 2017 Por

In this post, we will play with Bro IDS as a client fingerprinting techniques exploration tool.

As is known, during the initial TLS handshake (used, among others, by HTTPS on web browsers), a message called ClientHello is exchanged. In this message, the client specifies the supported cryptographic primitives (the so-called cipher suites).

For example, Firefox 50.1.0 under Linux sends a ClientHello like this, as shown with the Wireshark dissector:

There are webpages that inform us about the TLS characteristics of our browser, such as Qualys SSL Labs SSL Client Test.

The interesting thing is that other type of TLS client will normally use a different set of ciphersuites. For example, for wget:

$ wget -qO/dev/null https://www.google.es

We have a handshake as follows:

As we can see, wget sends 68 different ciphersuites (insted of Firefox’ 15 suites), and their order is also different.

Therefore, in addition to other parameters, such as the TLS version used (which is also present in the messages), the number of ciphersuites and their order in the ClientHello message provide us a fingerprint of the client.

Firefox will generate a different fingerprint than Chrome, and in the same way, the fingerprint of the Tor client (if we don’t use it with pluggable transports) will be different of those of a Python script, Microsoft web request APIs or a RAT coded in Delphi…

This is a fact that we can take advantage of to idenfity unathorized applications or to detect malware that does not bother to hide in plain sight as a HTTPS browser used inside the organization for its command and control or exfiltration activities ;-)

Enter Bro

It’s not rocket science to parse the ClientHello messages; there are p0f extensions and ad hoc tools such as FingerprinTLS. As expected, Bro IDS is also able to dissect the TLS protocol; we can find the constants used in the file /usr/local/bro/share/bro/base/protocols/ssl/consts.bro.

The magic numbers of the TLS versions and the mapping with the corresponding names are:

[...]
        const SSLv2  = 0x0002;
        const SSLv3  = 0x0300;
        const TLSv10 = 0x0301;
        const TLSv11 = 0x0302;
        const TLSv12 = 0x0303;
        const TLSv13 = 0x0304;
[...]
        const version_strings: table[count] of string = {
                [SSLv2] = "SSLv2",
                [SSLv3] = "SSLv3",
                [TLSv10] = "TLSv10",
                [TLSv11] = "TLSv11",
                [TLSv12] = "TLSv12",
                [TLSv13] = "TLSv13",
                [DTLSv10] = "DTLSv10",
                [DTLSv12] = "DTLSv12"
        } &default=function(i: count):string
[...]

In the same way, a little later in the file, we have the different ciphersuites and the cipher_desc data structure:

[...]
        const TLS_RSA_WITH_3DES_EDE_CBC_SHA = 0x000A;
        const TLS_DH_DSS_EXPORT_WITH_DES40_CBC_SHA = 0x000B;
        const TLS_DH_DSS_WITH_DES_CBC_SHA = 0x000C;
        const TLS_DH_DSS_WITH_3DES_EDE_CBC_SHA = 0x000D;
        const TLS_DH_RSA_EXPORT_WITH_DES40_CBC_SHA = 0x000E;
[...]
        const cipher_desc: table[count] of string = {
                [SSLv20_CK_RC4_128_EXPORT40_WITH_MD5] =
                        "SSLv20_CK_RC4_128_EXPORT40_WITH_MD5",
                [SSLv20_CK_RC4_128_WITH_MD5] = "SSLv20_CK_RC4_128_WITH_MD5",
                [SSLv20_CK_RC2_128_CBC_WITH_MD5] = "SSLv20_CK_RC2_128_CBC_WITH_MD5",
                [SSLv20_CK_RC2_128_CBC_EXPORT40_WITH_MD5] =
                        "SSLv20_CK_RC2_128_CBC_EXPORT40_WITH_MD5",
[...]

As we saw in the previous post, when an interesting fact occurs, Bro “shoots” an event, that in the case of a ClientHello message is (/usr/local/bro/share/bro/base/bif/plugins/Bro_SSL.events.bif.bro):

global ssl_client_hello: event(
  c: connection,
  version: count,
  possible_ts: time,
  client_random: string,
  session_id: string,
  ciphers: index_vec );

So we already have all the ingredients to execute fingerprinting. The script would look something like:

event ssl_client_hello (c: connection,
    version: count,
    possible_ts: time,
    client_random: string,
    session_id: string,
    ciphers: index_vec)
{
    local ciphers_str: vector of string;

    for (i in ciphers) {
        ciphers_str[i] = SSL::cipher_desc[ciphers[i]];
    }

    print fmt("%s %s", SSL::version_strings[version],
        join_string_vec(ciphers_str, ","));
}

The ciphers vector present in the event prototype stores the indexes with which it is possible to access the different ciphersuites contained in the message, which we map to their corresponding text strings using the cipher_desc table. To show them separated by a comma, we use the join_string_vec() function, similar to the join() function/method of Perl, Python, and other languages. If we save the script in the log_cipher_suites.bro file, we can perform a quick test by passing it on a previously captured tls.pcap file:

$ /usr/local/bro/bin/bro -C -r tls.pcap log_cipher_suites.bro
TLSv12 TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA,TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA,TLS_DHE_RSA_WITH_AES_128_CBC_SHA,TLS_DHE_RSA_WITH_AES_256_CBC_SHA,TLS_RSA_WITH_AES_128_CBC_SHA,TLS_RSA_WITH_AES_256_CBC_SHA,TLS_RSA_WITH_3DES_EDE_CBC_SHA
TLSv12 TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA,TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA,TLS_DHE_RSA_WITH_AES_128_CBC_SHA,TLS_DHE_RSA_WITH_AES_256_CBC_SHA,TLS_RSA_WITH_AES_128_CBC_SHA,TLS_RSA_WITH_AES_256_CBC_SHA,TLS_RSA_WITH_3DES_EDE_CBC_SHA

And there we have the 15 ciphersuites sent by Firefox, that uses TLSv1.2.

The logging framework

The print works, and all will be stored in stdout.log when Bro runs as a daemon, but… what if we want to send the output to another file, or to Elasticsearch? For that Bro provides the logging framework. Its usage is very simple (once you’ve got the catch, of course ;-)

To transform the previous “test” script into the final version, first we create a module called TlsFingerprint:

module TlsFingerprint;

Next, we have to define two variables that the module will export to the rest of Bro. By convention, they are called LOG and Info:

export {
    redef enum Log::ID += { LOG };

    type Info: record {
        ts: time &log;
        uid: string &log;
        id: conn_id &log;
        tls_version: string &log;
        ciphers: vector of string &log;
    };
}

As we can see, Info is the entry that will be recorded in the file. Each log entry will contain the timestamp (in the usual UN*X epoch format), a pair of identifiers, the TLS version and the different ciphersuites. The record is defined using Bro data types, such as vector of string, and the framework will be in charge of automatically storing it in a text file, Elasticsearch, or whatever backend we have configured.

To specify the name of the log file, we hook the initialization of Bro (event bro_init()) and we call the create_stream() function of the Log module (where resides the logging framework code):

event bro_init() &priority=5
{
    Log::create_stream(TlsFingerprint::LOG, [$columns=Info, $path="tls_finger"]);
}

The file will therefore be called tls_finger.log, and it will reside within the Bro log directories. It will be rotated automatically.

Finally, the new code for the ssl_client_hello() event is as follows:

event ssl_client_hello (c: connection,
    version: count,
    possible_ts: time,
    client_random: string,
    session_id: string,
    ciphers: index_vec)
{
    local ciphers_str: vector of string;
    local rec: TlsFingerprint::Info;

    for (i in ciphers) {
        ciphers_str[i] = SSL::cipher_desc[ciphers[i]];
    }

    rec$ts = network_time();
    rec$uid = c$uid;
    rec$id = c$id;
    rec$tls_version = SSL::version_strings[version];
    rec$ciphers = ciphers_str;

    Log::write(TlsFingerprint::LOG, rec);
}

As we see, we defined the rec variale as the type of record that we declared before (TlsFingerprint::Info), we fill it with the corresponding values values, and finally we call the function Log::write(). Bro already “knows” how to represent data types such as connection or vector of string in the log file or the corresponding backend, separating the fields (source IP address, source port, destination IP address, destination port, etc.) as appropriate.

The final tls_finger.bro would be as follows:

module TlsFingerprint;

export {
    redef enum Log::ID += { LOG };

    type Info: record {
        ts: time &log;
        uid: string &log;
        id: conn_id &log;
        tls_version: string &log;
        ciphers: vector of string &log;
    };
}

event bro_init() &priority=5
{
    Log::create_stream(TlsFingerprint::LOG, [$columns=Info, $path="tls_finger"]);
}

event ssl_client_hello (c: connection,
    version: count,
    possible_ts: time,
    client_random: string,
    session_id: string,
    ciphers: index_vec)
{
    local ciphers_str: vector of string;
    local rec: TlsFingerprint::Info;

    for (i in ciphers) {
        ciphers_str[i] = SSL::cipher_desc[ciphers[i]];
    }

    rec$ts = network_time();
    rec$uid = c$uid;
    rec$id = c$id;
    rec$tls_version = SSL::version_strings[version];
    #rec$ciphers = join_string_vec(ciphers_str, ",");
    rec$ciphers = ciphers_str;

    Log::write(TlsFingerprint::LOG, rec);
}

If we put it inside the /usr/local/share/bro/site dir, we can load it from local.bro with the following line:

@load site/tls_finger

After the corresponding broctl deploy, the log file will be populated with entries such as:

#fields ts      uid     id.orig_h       id.orig_p       id.resp_h       id.resp_p       tls_version     ciphers
#types  time    string  addr    port    addr    port    string  vector[string]
1485109320.798053  CxDC4MuavmTCRktJ3  192.168.1.1  52659  192.168.1.2  443  TLSv10  TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA,TLS_DHE_RSA_WITH_AES_256_CBC_SHA,TLS_DHE_RSA_WITH_AES_128_CBC_SHA,TLS_RSA_WITH_AES_256_CBC_SHA,TLS_RSA_WITH_AES_128_CBC_SHA,TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA,TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA,TLS_DHE_DSS_WITH_AES_256_CBC_SHA,TLS_DHE_DSS_WITH_AES_128_CBC_SHA,TLS_RSA_WITH_3DES_EDE_CBC_SHA,TLS_DHE_DSS_WITH_3DES_EDE_CBC_SHA,TLS_RSA_WITH_RC4_128_SHA,TLS_RSA_WITH_RC4_128_MD5

As we keep getting more info, it’s time to analyze it in search of anomalies ;-) For example, to sort the sets of ciphersuites from least to most used:

$ cat tls_finger.log | awk -F"\t" '{ print $8 }' | sort | uniq -c | sort -n

Of course, well-done, quality malware will try to hide in plain sight, impersonating the TLS fingerprint of the web browser normally used in the pwned workstation, but perhaps we get to hunt some cool stuff ;-)

Best regards! @pmarinram

See also in:

Comments

  1. Jon Zeolla says
    5 de February de 2017 at 9:21 pm

    Did you consider trying to infer clients and then log information about them using the software framework?

  2. Pablo says
    6 de February de 2017 at 9:11 am

    Yes, it would be a nice next step!