GoSpider is a Fast web spider written in Go. Installation go get -u github.com/jaeles-project/gospider Features Fast web crawling Brute force and parse sitemap.xml Parse robots.txt Generate and verify link from JavaScript files Link Finder Find AWS-S3 from response source Find subdomains from response source Get URLs from Wayback Machine, Common Crawl, Virus Total, Alien Vault …