If the other documentation left you wondering what to do with the results of your scans, and unsure how to get rid of those pesky leaked secrets, then look no further!
An End-to-End example walkthrough of a tartufo
scan and the process of purging the dirty evil passwords that somehow ended up in your code commits.
Select and clone the repo you want to run tartufo on
Clone your repo, variables used later:
GITHUBPROJECT="yourproject"
GITHUBREPO="myrepo.git"
GITHUBADDRESS="github.com"
git clone --mirror git@${GITHUBADDRESS}:${GITHUBPROJECT}/${GITHUBREPO}
2. Use tartufo
to scan your repository and find any secrets in its history!
Scan your repo!
Run Tartufo on your repo and create a list of high entropy items to remove:
tartufo --regex --output-format json scan-local-repo ${GITHUBREPO} | \
jq -r '.found_issues[].matched_string' | \
sort -u > remove.txt
Now you have a “bad password” file! Take a look through it, see if anything is wrong. This file will be used by BFG
to replace these flagged “bad password” entries with ***REMOVED***
.
3. Cleanup repo using BFG
and the above remove.txt file
There’s a very slick tool designed to clean up git commit history called BFG. By default, BFG
doesn’t modify the contents of your latest commit on your main (or ‘HEAD’) branch, even though it will clean all the commits before it.
This of course means if you have active code with “bad passwords”, tartufo
will still fail. But let’s take the bulk of the old entries out first.
Cleanup with BFG
wget https://repo1.maven.org/maven2/com/madgag/bfg/1.13.2/bfg-1.13.2.jar
Make a backup
cp -r ${GITHUBREPO} backup_${GITHUBREPO}
java -jar bfg-1.13.2.jar --replace-text remove.txt ${GITHUBREPO}
4. Uh Oh!
Occasionally the results will be too big to process all at once. If that happens, simply split up the results and loop through them.
occasionally the results will be to big to process all at once
split -l 200 remove.txt
for f in x*; do java -jar bfg-1.13.2.jar --replace-text $f ${GITHUBREPO}; done
5. Proceed with cleanup/audit
Now you have removed the low hanging fruit, it’s time to look at the tough stuff
run tartufo again to check for any remaining potential secrets
leftovers=`tartufo --regex -od ~/temp scan-local-repo ${GITHUBREPO}`
tmppath=`echo -e "$leftovers" | tail -n1 | awk '{print $6}'`
look through the remaining strings
if there's anything that looks like it shouldn't be there, dig into it and clear it out
cat ${tmppath}/* | jq '. | " \(.file_path) \(.matched_string) \(.signature)"' | sort -u
For more onformation click here.
Introduction to the Model Context Protocol (MCP) The Model Context Protocol (MCP) is an open…
While file extensions in Linux are optional and often misleading, the file command helps decode what a…
The touch command is one of the quickest ways to create new empty files or update timestamps…
Handling large numbers of files is routine for Linux users, and that’s where the find command shines.…
Managing files and directories is foundational for Linux workflows, and the mv (“move”) command makes it easy…
Creating directories is one of the earliest skills you'll use on a Linux system. The mkdir (make…