Kali Linux

Octopii : An AI-powered Personal Identifiable Information (PII) Scanner

Octopii is an open-source AI-powered Personal Identifiable Information (PII) scanner that can look for image assets such as Government IDs, passports, photos and signatures in a directory.

Working

Octopii uses Tesseract’s Optical Character Recognition (OCR) and Keras’ Convolutional Neural Networks (CNN) models to detect various forms of personal identifiable information that may be leaked on a publicly facing location. This is done in the following steps:

1. Importing and cleaning image(s)

The image is imported via OpenCV and Python Imaging Library (PIL) and is cleaned, deskewed and rotated for scanning.

2. Performing image classification and Optical Character Recognition (OCR)

A directory is looped over and searched for images. These images are scanned for unique features via the image classifier (done by comparing it to a trained model), along with OCR for finding substrings within the image. This may have one of the following outcomes:

  • Best case (score >=90): The image is sent into the image classifier algorithm to be scanned for features such as an ISO/IEC 7810 card specification, colors, location of text, photos, holograms etc. If it is successfully classified as a type of PII, OCR is performed on it looking for particular words and strings as a final check. When both of these are confirmed, the result from Octopii is extremely reliable.
  • Average case (score >=50): The image is partially/incorrectly identified by the image classifier algorithm, but an OCR check finds contradicting substrings and reclassifies it.
  • Worst case (score >=0): The image is only identified by the image classifier algorithm but an OCR scan returns no results.
  • Incorrect classification: False positives due to a very small model or OCR list may incorrectly classify PIIs, giving inaccurate results.

As a final verification method, images are scanned for certain strings to verify the accuracy of the model.

The accuracy of the scan can determined via the confidence scores in output. If all the mentioned conditions are met, a score of 100.0 is returned.

To train the model, data can also be fed into the model_generator.py script, and the newly improved h5 file can be used.

Usage

  1. Install all dependencies via pip install -r requirements.txt.
  2. Install the Tesseract helper locally via sudo apt install tesseract-ocr -y (for Ubuntu/Debian).
  3. To run Octopii, type python3 octopii.py <location name>, for example python3 octopii.py pii_list/

python3 octopii.py <location to scan> <additional flags>

Octopii currently supports local scanning and scanning S3 directories and open directory listings via their URLs.

Example

owais@artemis ~ $ python3 octopii.py pii_list

Not a valid image format: pii_list/aadhaar/aadhaar-8.gif

[
    {
        "asset_type": Credit and Debit Cards,
        "country_of_origin": "International",
        "confidence": 100,
        "file_name": "credit-card.jpg",
        "extension": "jpg",
        "path": "https://pii-carbonconsole.fra1.digitaloceanspaces.com/credit-card.jpg"
    },
    {
        "asset_type": "PAN",
        "country_of_origin": "IN",
        "confidence": 100,
        "file_name": "dummy-PAN-India.jpg",
        "extension": "jpg",
        "path": "https://pii-carbonconsole.fra1.digitaloceanspaces.com/dummy-PAN-India.jpg"
    },
    {
        "asset_type": Aadhaar,
        "country_of_origin": "IN",
        "confidence": 100,
        "file_name": "dummy-aadhaar.jpg",
        "extension": "jpg",
        "path": "https://pii-carbonconsole.fra1.digitaloceanspaces.com/dummy-aadhaar.jpg"
    },
    {
        "asset_type": Driver License,
        "country_of_origin": "International",
        "confidence": 100,
        "file_name": "dummy-drivers-license-nebraska-us.jpg",
        "extension": "jpg",
        "path": "https://pii-carbonconsole.fra1.digitaloceanspaces.com/dummy-drivers-license-nebraska-us.jpg"
    },
    {
        "asset_type": Passport,
        "country_of_origin": "International",
        "confidence": 100,
        "file_name": "dummy-passport-britain.jpg",
        "extension": "jpg",
        "path": "https://pii-carbonconsole.fra1.digitaloceanspaces.com/dummy-passport-britain.jpg"
    },
    {
        "asset_type": Passport,
        "country_of_origin": "International",
        "confidence": 100,
        "file_name": "dummy-passport-india.jpg",
        "extension": "jpg",
        "path": "https://pii-carbonconsole.fra1.digitaloceanspaces.com/dummy-passport-india.jpg"
    },
    {
        "asset_type": "Signature",
        "country_of_origin": null,
        "confidence": 7,
        "file_name": "dummy-signature.png",
        "extension": "png",
        "path": "https://pii-carbonconsole.fra1.digitaloceanspaces.com/dummy-signature.png"
    }
]
R K

Recent Posts

Understanding the Model Context Protocol (MCP) and How It Works

Introduction to the Model Context Protocol (MCP) The Model Context Protocol (MCP) is an open…

1 day ago

The file Command – Quickly Identify File Contents in Linux

While file extensions in Linux are optional and often misleading, the file command helps decode what a…

2 days ago

How to Use the touch Command in Linux

The touch command is one of the quickest ways to create new empty files or update timestamps…

2 days ago

How to Search Files and Folders in Linux Using the find Command

Handling large numbers of files is routine for Linux users, and that’s where the find command shines.…

2 days ago

How to Move and Rename Files in Linux with the mv Command

Managing files and directories is foundational for Linux workflows, and the mv (“move”) command makes it easy…

2 days ago

How to Create Directories in Linux with the mkdir Command

Creating directories is one of the earliest skills you'll use on a Linux system. The mkdir (make…

2 days ago