crawlergo is a browser crawler that uses chrome headless mode for URL collection. It hooks key positions of the whole web page…