Hfinger – Fingerprinting Malware HTTP Requests

Tool for fingerprinting HTTP requests of malware. Based on Tshark and written in Python3. Working prototype stage.

Its main objective is to provide unique representations (fingerprints) of malware requests, which help in their identification. 

Unique means here that each fingerprint should be seen only in one particular malware family, yet one family can have multiple fingerprints.

Hfinger represents the request in a shorter form than printing the whole request, but still human interpretable.

Hfinger can be used in manual malware analysis but also in sandbox systems or SIEMs.

The generated fingerprints are useful for grouping requests, pinpointing requests to particular malware families, identifying different operations of one family, or discovering unknown malicious requests omitted by other security systems but which share fingerprint.

An academic paper accompanies work on this tool, describing, for example, the motivation of design choices, and the evaluation of the tool compared to p0f, FATT, and Mercury.

Table Of Contents

1. The idea
2. Installation
3. Usage
4. Fingerprint creation
5. Report modes

The Idea

The basic assumption of this project is that HTTP requests of different malware families are more or less unique, so they can be fingerprinted to provide some sort of identification.

Hfinger retains information about the structure and values of some headers to provide means for further analysis.

For example, grouping of similar requests – at this moment, it is still a work in progress.

After analysis of malware’s HTTP requests and headers, we have identified some parts of requests as being most distinctive. These include:

• Request method
• Protocol version
• Header order
• Popular headers’ values
• Payload length, entropy, and presence of non-ASCII characters

Additionally, some standard features of the request URL were also considered. All these parts were translated into a set of features, described in details here.

The above features are translated into varying length representation, which is the actual fingerprint.

Depending on report mode, different features are used to fingerprint requests. More information on these modes is presented below.

The feature selection process will be described in the forthcoming academic paper.

Installation

Minimum requirements needed before installation:

• Python >= 3.3,
• Tshark >= 2.2.0.

Installation available from PyPI:

pip install hfinger

Hfinger has been tested on Xubuntu 22.04 LTS with tshark package in version 3.6.2, but should work with older versions like 2.6.10 on Xubuntu 18.04 or 3.2.3 on Xubuntu 20.04.

Please note that as with any PoC, you should run Hfinger in a separated environment, at least with Python virtual environment. Its setup is not covered here, but you can try this tutorial.

Usage

After installation, you can call the tool directly from a command line with hfinger or as a Python module with python -m hfinger.

For example:

foo@bar:~$ hfinger -f /tmp/test.pcap
[{"epoch_time": "1614098832.205385000", "ip_src": "127.0.0.1", "ip_dst": "127.0.0.1", "port_src": "53664", "port_dst": "8080", "fingerprint": "2|3|1|php|0.6|PO|1|us-ag,ac,ac-en,ho,co,co-ty,co-le|us-ag:f452d7a9/ac:as-as/ac-en:id/co:Ke-Al/co-ty:te-pl|A|4|1.4"}]

For more information click here.