OnionSearch is a powerful Python3 tool designed for scraping URLs from various “.onion” search engines on the dark web.
Ideal for researchers and cybersecurity professionals, this script offers a streamlined approach to accessing hidden services.
With support for multiple search engines and customizable search parameters, OnionSearch is an essential utility for deep web exploration.
pip3 install onionsearc
git clone https://github.com/megadose/OnionSearch.git
cd OnionSearch/
python3 setup.py install
usage: onionsearch [-h] [--proxy PROXY] [--output OUTPUT]
[--continuous_write CONTINUOUS_WRITE] [--limit LIMIT]
[--engines [ENGINES [ENGINES ...]]]
[--exclude [EXCLUDE [EXCLUDE ...]]]
[--fields [FIELDS [FIELDS ...]]]
[--field_delimiter FIELD_DELIMITER] [--mp_units MP_UNITS]
search
positional arguments:
search The search string or phrase
optional arguments:
-h, --help show this help message and exit
--proxy PROXY Set Tor proxy (default: 127.0.0.1:9050)
--output OUTPUT Output File (default: output_$SEARCH_$DATE.txt), where $SEARCH is replaced by the first chars of the search string and $DATE is replaced by the datetime
--continuous_write CONTINUOUS_WRITE
Write progressively to output file (default: False)
--limit LIMIT Set a max number of pages per engine to load
--engines [ENGINES [ENGINES ...]]
Engines to request (default: full list)
--exclude [EXCLUDE [EXCLUDE ...]]
Engines to exclude (default: none)
--fields [FIELDS [FIELDS ...]]
Fields to output to csv file (default: engine name link), available fields are shown below
--field_delimiter FIELD_DELIMITER
Delimiter for the CSV fields
--mp_units MP_UNITS Number of processing units (default: core number minus 1)
[...]
By default, the script will run with the parameter mp_units = cpu_count() - 1
. It means if you have a machine with 4 cores, it will run 3 scraping functions in parallel.
You can force mp_units
to any value but it is recommended to leave to default. You may want to set it to 1 to run all requests sequentially (disabling multi-processing feature).
Please note that continuous writing to csv file has not been heavily tested with multiprocessing feature and therefore may not work as expected.
Please also note that the progress bars may not be properly displayed when mp_units
is greater than 1. It does not affect the results, so don’t worry.
To request all the engines for the word “computer”:
onionsearch "computer"
For more information click here.
An innovative tool designed to revolutionize the testing, evaluation, and security of LLM applications. This…
A sophisticated tool designed for exploiting vulnerabilities in Android devices. This article dives into the…
SeamlessPass is a tool designed to obtain Microsoft 365 access tokens using on-premises Active Directory…
Comprehensive guide to the tools and resources pivotal in the world of forensic analysis. From…
An advanced Nmap script designed to detect potential vulnerabilities in network services. This article delves…
A powerful tool designed to tunnel TCP connections through a file. Ideal for circumventing firewalls…