NFStream : A Flexible Network Data Analysis Framework

NFStream is a Python package providing fast, flexible, and expressive data structures designed to make working with online or offline network data both easy and intuitive.

It aims to be the fundamental high-level building block for doing practical, real world network data analysis in Python. Additionally, it has the broader goal of becoming a common network data processing framework for researchers providing data reproducibility across experiments.

Main Features

  • Performance: nfstream is designed to be fast (x10 faster with pypy3 support) with a small CPU and memory footprint.
  • Layer-7 visibility: nfstream deep packet inspection engine is based on nDPI. It allows nfstream to perform reliable encrypted applications identification and metadata extraction (e.g. TLS, QUIC, TOR, HTTP, SSH, DNS).
  • Flexibility: add a flow feature in 2 lines as an NFPlugin.
  • Machine Learning oriented: add your trained model as an NFPlugin.

How to use it?

  • Dealing with a big pcap file and just want to aggregate it as network flows? nfstream make this path easier in few lines:

from nfstream import NFStreamer
my_awesome_streamer = NFStreamer(source=”facebook.pcap”) # or network interface (source=”eth0″)
for flow in my_awesome_streamer:
print(flow) # print it, append to pandas Dataframe or whatever you want :)!

NFEntry(
id=0,
first_seen=1472393122365,
last_seen=1472393123665,
version=4,
src_port=52066,
dst_port=443,
protocol=6,
vlan_id=0,
src_ip=’192.168.43.18′,
dst_ip=’66.220.156.68′,
total_packets=19,
total_bytes=5745,
duration=1300,
src2dst_packets=9,
src2dst_bytes=1345,
dst2src_packets=10,
dst2src_bytes=4400,
expiration_id=0,
master_protocol=91,
app_protocol=119,
application_name=’TLS.Facebook’,
category_name=’SocialNetwork’,
client_info=’facebook.com’,
server_info=’*.facebook.com’,
j3a_client=’bfcc1a3891601edb4f137ab7ab25b840′,
j3a_server=’2d1eb5817ece335c24904f516ad5da12′
)

  • From pcap to Pandas DataFrame?

import pandas as pd
streamer_awesome = NFStreamer(source=’devil.pcap’)
data = []
for flow in streamer_awesome:
data.append(flow.to_namedtuple())
my_df = pd.DataFrame(data=data)
my_df.head(5) # Enjoy!

  • Didn’t find a specific flow feature? add a plugin to nfstream in few lines:

from nfstream import NFPlugin

class my_awesome_plugin(NFPlugin):
def on_update(self, obs, entry):
if obs.length >= 666:
entry.my_awesome_plugin += 1

streamer_awesome = NFStreamer(source=’devil.pcap’, plugins=[my_awesome_plugin()])
for flow in streamer_awesome:
print(flow.my_awesome_plugin) # see your dynamically created metric in generated flows

Also Read – Blinder : A Python Library To Automate Time-Based Blind SQL Injection

Prerequisites

apt-get install libpcap-dev

Installation

Using pip

Binary installers for the latest released version are available:

pip3 install nfstream

Build from source

If you want to build nfstream on your local machine:

git clone https://github.com/aouinizied/nfstream.git
cd nfstream
python3 setup.py install

R K

Recent Posts

Pystinger : Bypass Firewall For Traffic Forwarding Using Webshell

Pystinger is a Python-based tool that enables SOCKS4 proxying and port mapping through webshells. It…

4 hours ago

CVE-Search : A Tool To Perform Local Searches For Known Vulnerabilities

Introduction When it comes to cybersecurity, speed and privacy are critical. Public vulnerability databases like…

5 hours ago

CVE-Search : A Tool To Perform Local Searches For Known Vulnerabilities

Introduction When it comes to cybersecurity, speed and privacy are critical. Public vulnerability databases like…

24 hours ago

How to Bash Append to File: A Simple Guide for Beginners

If you are working with Linux or writing bash scripts, one of the most common…

1 day ago

Mastering the Bash Case Statement with Simple Examples

What is a bash case statement? A bash case statement is a way to control…

1 day ago

How to Check if a File Exists in Bash – Simply Explained

Why Do We Check Files in Bash? When writing a Bash script, you often work…

3 days ago