Categories: Kali Linux

Galer : A Fast Tool To Fetch URLs From HTML Attributes By Crawl-In

Galer is a fast tool to fetch URLs from HTML attributes by crawl-in. Inspired by the @omespino Tweet, which is possible to extract src, href, url and action values by evaluating JavaScript through Chrome DevTools Protocol.

Installation

From Binary

The installation is easy. You can download a prebuilt binary from releases page, unpack and run! or with

(sudo) curl -sSfL https://git.io/galer | sh -s — -b /usr/local/bin

From Source

If you have go1.15+ compiler installed and configured:

GO111MODULE=on go get github.com/dwisiswant0/galer

From GitHub

git clone https://github.com/dwisiswant0/galer
cd galer
go build .
(sudo) mv galer /usr/local/bin

Usage

Basic Usage
- Simply, galer can be run with:

galer -u “http://domain.tld”

Flags

galer -h

This will display help for the tool. Here are all the switches it supports.

Flag	Description
-u, –url	Target to fetches (single target URL or list)
-e, –extension	Show only certain extensions (comma-separated, e.g. js,php)
-c, –concurrency	Concurrency level (default: 50)
–in-scope	Show in-scope URLs/same host only
-o, –output	Save fetched URLs output into file
-t, –timeout	Maximum time (seconds) allowed for connection (default: 60)
-s, –silent	Silent mode (suppress an errors)
-v, –verbose	Verbose mode show error details unless you weren’t use silent
-h, –help	Display its helps

Examples

Single URL

galer -u “http://domain.tld”

URLs from list

galer -u /path/to/urls.txt

From Stdin

cat urls.txt | galer

In case you want to chained with other tools:

subfinder -d domain.tld -silent | httpx -silent | galer

Library

You can use galer as library.

go get github.com/dwisiswant0/galer/pkg/galer

For example:

package main

import (
 "fmt"

 "github.com/dwisiswant0/galer/pkg/galer"
)

func main() {
 cfg := &galer.Config{
  Timeout: 60,
 }
 cfg = galer.New(cfg)

 run, err := cfg.Crawl("https://twitter.com")
 if err != nil {
  panic(err)
 }

 for _, url := range run {
  fmt.Println(url)
 }
}

TODOs