Duplicut is a modern password wordlist creation usually implies concatenating multiple data sources.
Ideally, most probable passwords should stand at start of the wordlist, so most common passwords are cracked instantly.
With existing dedupe tools you are forced to choose if you prefer to preserve the order OR handle massive wordlists.
Unfortunately, wordlist creation requires both:
So i wrote duplicut in highly optimized C to address this very specific need
Quick start
git clone https://github.com/nil0x42/duplicut
cd duplicut/ && make
./duplicut wordlist.txt -o clean-wordlist.txt
Options
Features
-l
option)-p
option)Implementation
Limitations
Technical Details
1- Memory optimized
An uint64
is enough to index lines in hashmap, by packing size
info within pointer’s extra bits:
2- Massive file handling
If whole file can’t fit in memory, it is split into virtual chunks, then each one is tested against next chunks.
So complexity is equal to th triangle number:
Throubleshotting
If you find a bug, or something doesn’t work as expected, please compile duplicut in debug mode and post an issue with attached output:
Starship is a powerful, minimal, and highly customizable cross-shell prompt designed to enhance the terminal…
Lemmy is an innovative, open-source platform designed for link aggregation and discussion, providing a decentralized…
The latest release of ImHex v1.37.0 introduces a host of exciting features and improvements, enhancing…
Ghauri is a cutting-edge, cross-platform tool designed to automate the detection and exploitation of SQL…
Writing tools have become indispensable for individuals looking to enhance their writing efficiency, accuracy, and…
PatchWerk is a proof-of-concept (PoC) tool designed to clean NTDLL syscall stubs by patching syscall…