Upgrading Tartufo – A Comprehensive Guide To Transitioning From Release 2 To Release 3

Upgrading tartufo from release 2 to release 3 introduces some behavioral and interface changes. Current users of release 2 should review this summary to understand how to transition to release 3 as painlessly as possible.

General Behavioral Changes

tartufo release 3 is generally more accurate than previous releases. It may detect problems that were not recognized by release 2 scans (especially earlier 2.x releases).

A scan of your code base prior to upgrading will simplify the process of identifying new findings that are attributable to these behavior changes so they can be remediated or suppressed.

Remote Repository Scanning

tartufo releases between 2.2.0 and 2.9.0 (inclusive) mishandled remote repositories. Only the repository’s default branch was scanned; secrets present only on other branches would not be discovered.

Additionally, the --branch branch-name option did not operate correctly. Some versions scanned nothing and reported no errors, and other versions aborted immediately after reporting the branch did not exist (even when it did).

tartufo release 3 scans all remote repository branches by default, and correctly scans only a single branch if one is specified using --branch. As a consequence, it may discover secrets that were not reported by earlier versions.

These fixes were backported to tartufo release 2.10.0.

Live Output

tartufo release 3 reports findings incrementally as a scan progresses; previous releases did not perform any reporting until the entire scan was completed.

Entropy Scanning

Beginning with release 3, tartufo recognizes base64url-encoded strings in addition to base64-encoded strings.

If your code contains base64url encodings (or strings that look like base64url encodings), these strings now will be checked for high entropy and may produce new findings.

Additionally, strings that contain combinations of base64 and base64url character sets (whether they are actual encodings or not) will be scanned differently by release 3.

Previously, base64 substrings would be extracted and scanned independently, but now the larger string will be scanned (once) in its entirety.

This can result in signature changes (because the new suspect string is larger than the string recognized by release 2.x) and possibly fewer findings (because one longer string will be flagged instead of multiple substrings).

Real-life files do not typically contain sequences that will exhibit this behavior.

Shallow Repositories

When tartufo release 2 scanned a shallow repository (a repository with no refs or branches found locally), it did not actually scan anything.

In the same situation, tartufo release 3 scans the repository HEAD as a single commit, effectively scanning the entire existing codebase (but none of its history) at once.

This scenario is commonly encountered in GitHub actions, which perform shallow checkouts.

Nonfunctional Options

tartufo release 3 uses pygit2 instead of GitPython to access git repositories.

While this provides vastly improved performance with generally equivalent functionality, some less-frequently used options require reimplementation and currently are nonfunctional. We plan to provide either replacements or reimplementations in the future.

The --since-commit option is intended to restrict scans to a subset of repository history; the --max-depth option provides roughly the same functionality specified differently.

Both options are ignored by tartufo release 3. Refer to #267 for more information about this topic.

Changes To Default Behavior

Some defaults have changed for the new release. If you wish to retain the previous behavior, adjust your configuration options to request it explicitly.

Regex Scanning

Previously, tartufo did not perform regex scanning for sensitive strings by default. Release 3 does perform regex scanning by default.

Explicitly disable regex scanning to preserve the old behavior:

[tool.tartufo]
regex = false

Alternatively, add --no-regex to your tartufo command line.

For more information click here.