FACT : A Tool To Collect, Process And Visualise Forensic Data From Clusters Of Machines

FACT is a tool to collect, process and visualise forensic data from clusters of machines running in the cloud or on-premise.

Deployment

For a basic single-node deployment, we recommend using Docker and Docker Compose. First, read docker-compose.yaml for configuration and requirements. Then, start the stack using:

docker-compose up -d

Installation

To install FACT for deployment

1. Docker Compose Single-node Deployment
2. Kubernetes Multi-node Deployment

For a development environment, see the developer documentation

Docker Compose Single-node Deployment

You can run FACT on platforms supported by Docker Desktop using our Docker Compose setup. This setup is not meant for production use yet.

Prerequisites

1. Docker
2. Docker Compose
3. A large amount of disk space for disk images
4. At least 4GB of RAM available, 8GB recommended

Configuring docker-compose.yml

Download docker-compose.yml from this repository.

Open it in a text editor and replace ${HOST_IP} with the IP address of your machine running Docker. Alternatively, set HOST_IP in your shell environment from which you run docker-compose. You can find your IP address using your operating system tools or when using Docker Desktop, try host.docker.internal.

The IP address can be either the one used to connect to the internet, or when using Docker Desktop, the internal IP address of the underlying Docker Desktop VM. It should be reachable from both your host and within Docker.

Starting all services

Open a shell, enter the directory containing docker-compose.yml and start all the services using:

docker-compose up -d

Using FACT

Check out the user guide.

User Guide

Concepts

• Controller: The overall orchestrator of the operations. Keeps a log of what the user has done and assigns tasks to workers.
• Workers: Take tasks from the controller. These tasks instruct the worker to capture and analyse evidence from targets.
• Targets: Machines that you wish to capture evidence from.

UI

Once you’ve installed FACT, you can open up the UI by opening http://localhost:3000 in a web browser to start using FACT.

Adding targets

The first thing you might want to do is to add a target. Visit the “Targets” page to add the targets you wish to capture evidence from into FACT. Give it a display name for your reference (and later on used in Kibana), and enter the user, host and port used to connect to the target.

You should add an existing SSH keypair to connect to the target, or generate a new one and install it on the target. Paste the private key into the box.

When using a non-root user, you should check the “Use sudo” checkbox.

At the moment, “Use sudo” only works when there is no password prompt (NOPASSWD) for the user.

Discovering disks

Before you can collect disks, FACT has to know about them. Select all the targets you’d like to capture and press “Scan disks”. Give it a while to discover the block devices on every machine.

Behind the scenes, this runs lsblk -lb on each target.

Capturing disk images

Once the disk discovery has completed, you can pick the disks you wish to capture. Picking the block device will capture the entire device including all its partitions. Picking a partition device will capture only the filesystem on that partition. Once you’re done picking, press “Capture disks”. This may take minutes, hours or days depending on the amount of data you are capturing, the number of workers you have and the disk performance of both the targets and the workers, amongst other factors.

Behind the scenes, this runs dd on each target and streams the device to the worker over SSH.

Analysing disk images

When disks have been captured, analysis can be started by selecting the devices and pressing “Ingest captures”. This will start the process of ingestion of the data within the collected disk images. It might take a while too.

Viewing the data

Once ingestion has completed, open Kibana by visiting http://localhost:5601.

To view a log of events, open the menu and visit “Discover”. You might be prompted to create an index pattern. Create an index pattern fact-*-files or fact-* to match all FACT indexes. Optionally, pick one of the file time attributes (mtime, ctime or atime) as the time field to sort entries by that attribute.

After your index pattern is created, you can open “Discover” again to view the log of events, search and sort through them.

Developing

See the developer documentation for details on setting up a development environment, linting and testing.

Developing FACT

Requirements

• Python
• Poetry
• Node.js
• Elasticsearch and Kibana
• A decent amount of disk space for disk images
• At least 4GB of RAM available, 8GB recommended

Setup

If you haven’t already, clone the repository.

Install development dependencies using:

Set up virtual environment and install dependencies
poetry install –no-root
Enter virtual environment
poetry shell
Install grpcwebproxy
python tools/download-grpcwebproxy.py
Optional: Install Git hooks to check your code and commit messages
python tools/hooks.py install
Optional: When working on the UI
cd ui
npm install

Checks

Before submitting commits or pull requests, consider running the code formatter, linter, type checks and tests:

With Git hooks installed
git commit
Without Git hooks installed
python tools/pre-commit.py

If you wish to skip the linter, type checks and tests:

With Git hooks installed
SKIP_CHECKS=y git commit
Without Git hooks installed
SKIP_CHECKS=y python tools/pre-commit.py

Tools

All the tools in tools/ must be run from the project root.

Writing Messages

We follow Conventional Commits 1.0.0 for commit messages and pull requests:

[optional scope]:

We use the following types:

• feat: A new feature
• fix: A bug fix
• docs: Documentation only changes
• style: Changes that do not affect the meaning of the code
• refactor: A code change that neither fixes a bug nor adds a feature
• test: Adding missing tests or correcting existing tests
• ci: Changes to our CI configuration files and scripts
• chore: Other changes that don’t modify src or test files
• wip: Work-in-progress

To help you write commit messages, you can use:

Asks you a series of questions about your commit
mkcommit –autoselect

Updating gRPC Protocol

After any change to the gRPC protocol as defined in the .proto files, the services, stubs and types must be regenerated using the protocol buffer compiler. This can be done by running:

Enter virtual environment if you haven’t already
poetry shell
Compile the files specified in Makefile if they changed
make proto
Do the same for the UI
make proto-ts