Katana Web Crawling for Security Reconnaissance

Katana is a next-generation web crawling and spidering framework developed by ProjectDiscovery. It helps security professionals map websites, discover URLs and identify endpoints that may not be visible through ordinary browsing. Its configurable crawling engine makes it useful for attack surface discovery, web application testing and automated reconnaissance.

The framework supports standard HTTP crawling and headless browser-based crawling. It can also parse JavaScript files, control crawl scope, filter results and export discovered endpoints for further analysis. These capabilities make Katana a practical addition to a web security testing workflow.

Key Features of Katana

• Fast web crawling: Configure crawl depth, request timeouts, concurrency and crawl duration to suit the target and testing requirements.
• Standard and headless modes: Use standard crawling for speed or headless crawling to inspect pages that depend on JavaScript rendering.
• JavaScript endpoint discovery: Parse JavaScript files to identify additional URLs and application endpoints.
• Scope control: Include approved domains and paths while excluding unrelated or restricted endpoints.
• Flexible input: Supply a single URL, a list of URLs or targets through standard input.
• Structured output: Save results as plain text or JSON Lines for processing by other tools.
• Custom request settings: Configure headers, cookies, proxies, resolvers and other crawling options.

How to Install Katana

Katana is written in Go. Before installing it, check the official Katana repository for the current Go version requirement. The project documentation currently specifies Go 1.26 or later for installation from source using the Go toolchain.

Install Katana with the following command:

CGO_ENABLED=1 go install github.com/projectdiscovery/katana/cmd/katana@latest

Make sure the Go binary directory is included in your system’s PATH. Confirm that the installation works by displaying the help menu:

katana -h

Precompiled releases and a Docker image are also available through the official project resources. Check the repository for platform-specific instructions and current installation options.

Basic Katana Usage

Start with a single website that you own or have permission to assess:

katana -u https://example.com

To increase the crawl depth, use the -d option. A deeper crawl can uncover more endpoints, but it may also increase request volume and execution time.

katana -u https://example.com -d 5

Katana can accept multiple URLs directly or read targets from a file. The following example loads URLs from targets.txt:

katana -list targets.txt

You can also pipe input from another command into Katana:

cat targets.txt | katana

Discover Endpoints with JavaScript and Headless Crawling

Modern web applications often expose routes and API endpoints through JavaScript files. Enable JavaScript crawling with -jc to expand endpoint discovery:

katana -u https://example.com -jc

For applications that rely on browser rendering, try headless mode. This mode uses browser-based processing and can discover content that standard HTTP crawling may miss.

katana -u https://example.com -hl

Headless crawling may require browser dependencies. Review the official documentation before configuring Chrome or other supported browser options. Automatic form filling is also available as an experimental feature, so use it cautiously in controlled test environments.

Control Scope and Filter Results

Scope restrictions help keep reconnaissance focused on authorized systems. Use -cs to specify crawl-scope patterns and -cos to exclude matching URLs. For example, the following command focuses crawling on URLs matching a login-related pattern:

katana -u https://example.com -cs login

To display only discovered URLs containing query parameters, use the -f field option:

katana -u https://example.com -f qurl -silent

Filtering can reduce noise and make it easier to identify paths, filenames and parameter names for manual review. For additional reconnaissance techniques, explore our guide to website OSINT tools and techniques.

Save and Export Crawl Results

Save discovered endpoints to a text file with the -o option:

katana -u https://example.com -o endpoints.txt

For machine-readable results, enable JSON Lines output with -j:

katana -u https://example.com -j -o endpoints.jsonl

These files can support endpoint review, asset inventory, and integration with other security tools. Review collected URLs before passing them into additional scanners, especially if the crawl includes query parameters or authenticated content.

Use Katana Responsibly

Katana is designed for security testing and automated reconnaissance, but aggressive crawling can generate significant traffic. Test only websites and applications you own or are explicitly authorized to assess. Keep crawl scope narrow, set reasonable request limits and avoid automated form interactions on production systems unless they are approved for testing.

Consult the official Katana documentation for current flags, advanced configuration and release updates, since command-line options can change between versions.