Kali Linux

ggshield : Detect secret in source code, scan your repo for leaks

ggshield is a CLI application that runs in your local environment or in a CI environment to help you detect more than 350+ types of secrets, as well as other potential security vulnerabilities or policy breaks.

ggshield uses our public API through py-gitguardian to scan and detect potential secrets on files and other text content.

Only metadata such as call time, request size and scan mode is stored from scans using ggshield, therefore secrets and policy breaks incidents will not be displayed on your dashboard and your files and secrets won’t be stored.

Installation

macOS

Using Homebrew

You can install ggshield using Homebrew by running the following command:

$ brew install gitguardian/tap/ggshield

Linux packages

Deb and RPM packages are available on Cloudsmith.

Setup instructions:

Deb packages
RPM packages

Other Operating Systems

Using pip

Install and update using pip:

$ pip install ggshield

ggshield supports Python 3.7 and newer.

The package should run on MacOS, Linux and Windows.

Updating

To update ggshield you can add the option -U/--upgrade to the pip install command

$ pip install -U ggshield

Initial setup

To use ggshield you need to authenticate against GitGuardian servers. To do so, use the ggshield auth login command. This command automates the provisioning of a personal access token and its configuration on the local workstation.

You can learn more about it from ggshield auth login documentation.

Alternatively, you can create your personal access token manually and you can store it in the GITGUARDIAN_API_KEY environment variable to complete the setup.

Once this is done, you can start scanning a repository with with ggshield secret scan repo /path/to/your/repo.

Command reference

Usage: ggshield [OPTIONS] COMMAND [ARGS]…
Options:
-c, –config-path FILE Set a custom config file. Ignores local and global
config files.
-v, –verbose Verbose display mode.
–allow-self-signed Ignore ssl verification.
–debug Show debug information.
–version Show the version and exit.
-h, –help Show this message and exit.
Commands:
api-status Show API status.
auth Commands to manage authentication.
install Install a pre-commit or pre-push git hook (local or global).
quota Show quotas overview.
secret Commands to work with secrets.

Secret scan commands

The ggshield secret scan command group contains the main ggshield commands, it has a few configuration options that can be used to override output behavior.

Usage: ggshield secret scan [OPTIONS] COMMAND [ARGS]…

Commands to scan various contents.
Options:
–json JSON output results [default: False]
–show-secrets Show secrets in plaintext instead of hiding
them.
–exit-zero Always return a 0 (non-error) status code, even
if incidents are found.The env var
GITGUARDIAN_EXIT_ZERO can also be used to set
this option.
–all-policies Present fails of all policies (Filenames,
FileExtensions, Secret Detection).By default,
only Secret Detection is shown.
-v, –verbose Verbose display mode.
-o, –output PATH Route ggshield output to file.
-b, –banlist-detector TEXT Exclude results from a detector.
–exclude PATH Do not scan the specified path.
–ignore-default-excludes Ignore excluded patterns by default. [default:
False]
-h, –help Show this message and exit.
Commands:
archive scan archive .
ci scan in a CI environment.
commit-range scan a defined COMMIT_RANGE in git.
docker scan a docker image .
path scan files and directories.
pre-commit scan as a pre-commit git hook.
pre-push scan as a pre-push git hook.
pre-receive scan as a pre-receive git hook.
pypi scan a pypi package .
repo scan a REPOSITORY’s commits at a given URL or path.

ggshield secret scan has different subcommands for each type of scan.

`secret scan ci`: scan each commit since the last build in your CI

Usage: ggshield secret scan ci [OPTIONS]
scan in a CI environment.
Options:
-h, –help Show this message and exit.

secret scan commit-range: scan each commit in the given commit range

Usage: ggshield secret scan commit-range [OPTIONS] COMMIT_RANGE
scan a defined COMMIT_RANGE in git.
git rev-list COMMIT_RANGE to list several commits to scan. example: ggshield
secret scan commit-range HEAD~1…
Options:
-h, –help Show this message and exit.

secret scan path: scan files or directories with the recursive option

Usage: ggshield secret scan path [OPTIONS] PATHS…
scan files and directories.
Options:
-r, –recursive Scan directory recursively
-y, –yes Confirm recursive scan
-h, –help Show this message and exit.

secret scan pre-commit: scan every changes that have been staged in a git repository

Usage: ggshield secret scan pre-commit [OPTIONS] [PRECOMMIT_ARGS]…
scan as a pre-commit git hook.
Options:
-h, –help Show this message and exit.

secret scan pypi: scan a pypi package

Usage: ggshield secret scan pypi [OPTIONS] PACKAGE_NAME
scan a pypi package .
Options:
-h, –help Show this message and exit.

Under the hood this command uses the pip download command to download the python package.

You can use pip environment variables or configuration files to set pip download parameters as explained in pip documentation.

For example, you can set pip --index-url parameter by setting PIP_INDEX_URL environment variable.

Configuration

Configuration in ggshield follows a global>local>CLI configuration scheme.

Meaning options on local overwrite or extend global and options on CLI overwrite or extend local.

ggshield will search for a global configuration file in the user’s home directory (example: ~/.gitguardian.yml on Linux and %USERPROFILE%\.gitguardian on Windows).

ggshield will recognize as well a local configuration file in the user’s working directory (example: ./.gitguardian.yml).

You can also use the option --config-path on the main command to set another configuration file. In this case, neither local nor global configuration files will be evaluated (example: ggshield --config-path=~/Desktop/only_config.yaml scan path -r .)

A sample configuration file can be found at .gitguardian.example

Migrating a v1 configuration file

If you have a v1 configuration file, you can run ggshield config migrate to let ggshield migrate it for you. The command modifies the configuration file in place, but it keeps the previous version as a .gitguardian.yaml.old file.

Alternatively, you can follow these steps to migrate your configuration file manually:

debug: false # default: false

Add a version: 2 entry.
If the configuration file contains an all-policies key, remove it: it’s no longer supported.
If the configuration file contains an ignore-default-excludes key, remove it: it’s no longer supported.
If the configuration file contains an api-url key, replace it with an instance key, pointing to the dashboard URL.
If the configuration file contains one of the following keys: paths-ignore, matches-ignore, show-secrets, banlisted-detectors:
- Create a secret key.
- Move paths-ignore to secret.ignored-paths.
- Move matches-ignore to secret.ignored-matches. If some match entries are strings instead of (name, match) objects, turn them into (name, match) objects.
- Move banlisted-detectors to secret.ignored-detectors.
- Move show-secrets to secret.show-secrets.

Environment Variables

ggshield can also be configured using environment variables.

Environment variables overrides settings set on your configuration file but are overridden by command line options.

At startup, ggshield attempts to load environment variables from different environment files in the following order:

path pointed to by the environment variable GITGUARDIAN_DOTENV_PATH
.env at your current work directory.
.env at the root of the current git directory

Only one file will be loaded of the three.

Reference of current environment variables supported by ggshield:

GITGUARDIAN_API_KEY: API Key for the GitGuardian API. Use this if you don’t want to use the ggshield auth commands.
GITGUARDIAN_INSTANCE: Custom URL of the GitGuardian dashboard. The API URL will be inferred from it.
GITGUARDIAN_API_URL: Custom URL for the scanning API. Deprecated, use GITGUARDIAN_INSTANCE instead.
GITGUARDIAN_DONT_LOAD_ENV: If set to any value, environment variables won’t be loaded from a file.
GITGUARDIAN_DOTENV_PATH: If set to a path, ggshield will attempt to load the environment from the specified file.
GITGUARDIAN_TIMEOUT: If set to a float, ggshield secret scan pre-receive will timeout after the specified value. Set to 0 to disable the timeout.
GITGUARDIAN_MAX_COMMITS_FOR_HOOK: If set to an int, ggshield secret scan pre-receive and ggshield secret scan pre-push will not scan more than the specified value of commits in a single scan.
GITGUARDIAN_CRASH_LOG: If set to True, ggshield will display a full traceback when crashing.

On-premises configuration

ggshield can be configured to run on your on-premise GitGuardian instance.

First, you need to point ggshield to your instance, by either defining the instance key in your .gitguardian.yaml configuration file or by defining the GITGUARDIAN_INSTANCE environment variable.

Then, you need to authenticate against your instance, by either using the ggshield auth login --instance https://mygitguardianinstance.mycorp.local command using the --instance option or by obtaining an API key from your dashboard administrator and storing it in the GITGUARDIAN_API_KEY environment variable.

Ignoring files

By default ggshield ignores certain files and directories.

This list can be found in ggshield/core/utils.py under IGNORED_DEFAULT_PATTERNS.

You can also add custom patterns to ignore by using the --exclude option or the key ignored-paths in your .gitguardian.yaml

Pre-push

In case there are more than a 50 commits in a push the hook will be skipped.

The amount of commits to scan before skipping the hook can be configured by the key max-commits-for-hook in ggshield configuration file.

Pre-push hooks are executed just before git push sends data to the remote host. It will pickup and scan the range of commits between the local ref and the origin ref.

If incidents are detected in this range the push will be cancelled.