Secure Your Repositories: Prevent Credential Leaks with Gitleaks

Leaking secrets is like leaving your front door wide open. Most won’t notice, but it only takes one bad actor to walk in and cause havoc.

Even a tiny leak can lead to a massive data breach. Consider these real-world examples:

Uber breach (2016)

One of the most catastrophic data breaches due to leaked secrets was the Uber breach in 2016. Hackers exposed the personal data of 57 million customers and drivers, including names, email addresses, and phone numbers.

The breach happened because attackers accessed Uber’s GitHub repository, where sensitive information, including AWS credentials, was stored. With these credentials, they gained access to Uber’s cloud servers and found a data archive containing personal information.

Equifax breach (2017)

The Equifax data breach exposed the personal information of 147 million people, including Social Security numbers, birth dates, and addresses. The breach occurred due to a vulnerability in a web application that Equifax failed to patch.

Additionally, sensitive data was stored in plaintext, and administrative credentials were compromised, allowing attackers to access and steal data over several months.

Capital One breach (2019)

In 2019, Capital One experienced a breach that exposed the personal information of over 100 million customers. A former AWS employee exploited a misconfigured firewall in Capital One’s cloud infrastructure, accessing to sensitive data stored on AWS S3.

The breach included Social Security numbers, bank account numbers, and credit scores. The incident highlighted the importance of securing cloud infrastructure and properly configuring access controls.

Adobe breach (2013)

Adobe experienced a massive data breach in 2013, affecting 38 million users. Hackers accessed Adobe’s servers and stole source code for several Adobe products, along with user information, including encrypted passwords and payment card details.

The breach occurred due to weak password storage practices and poor security controls. It demonstrated the risks of storing sensitive information without adequate encryption and access controls.

LinkedIn breach (2012)

In 2012, LinkedIn suffered a data breach that exposed the passwords of approximately 6.5 million users. The passwords were stored using a weak hashing algorithm without proper salt, making them easy to crack.

How do secrets leak?

1. Hardcoded Values: Putting sensitive data directly in your code is risky. It happens more often than you’d think.
2. Version Control: Even if you remove a secret, it might still exist in your Git history.
3. Misconfigured Access: Accidentally exposing a private repository or granting too much access can spill your secrets.

Preventive measures

1. Use Environment Variables: Keep sensitive information out of your codebase by storing it in environment variables.
2. Regularly Scan Your Codebase: Regular scans can help catch potential leaks before they become a problem.
3. Audit Your Git History: Regularly audit and clean up your Git history. Tools like git filter-repo can help remove sensitive data from past commits.
4. Secure Access: Limit who can access your repositories and use two-factor authentication to add an extra layer of security

Gitleaks

Gitleaks is an open-source, free tool that I have no affiliation with. It’s freely available and easy to use, making it a great choice for regular security checks.

1
# MacOS
2
brew install gitleaks
3

4
# Docker (DockerHub)
5
docker pull zricethezav/gitleaks:latest
6
docker run -v ${path_to_host_folder_to_scan}:/path zricethezav/gitleaks:latest [COMMAND] --source="/path" [OPTIONS]
7

8
# Docker (ghcr.io)
9
docker pull ghcr.io/gitleaks/gitleaks:latest
10
docker run -v ${path_to_host_folder_to_scan}:/path ghcr.io/gitleaks/gitleaks:latest [COMMAND] --source="/path" [OPTIONS]
11

12
# From Source (make sure `go` is installed)
13
git clone https://github.com/gitleaks/gitleaks.git
14
cd gitleaks
15
make build

Scanning Gitleaks repository with Gitleaks

To demonstrate Gitleaks’ effectiveness, I ran a scan on its own repository.

1
➜  git clone [email protected]:gitleaks/gitleaks.git
2
Cloning into 'gitleaks'...
3
remote: Enumerating objects: 8623, done.
4
remote: Counting objects: 100% (28/28), done.
5
remote: Compressing objects: 100% (23/23), done.
6
remote: Total 8623 (delta 11), reused 17 (delta 5), pack-reused 8595
7
Receiving objects: 100% (8623/8623), 5.32 MiB | 5.44 MiB/s, done.
8
Resolving deltas: 100% (4688/4688), done.
9
➜ gitleaks detect --source ./gitleaks
10

11
    ○
12
    │╲
13
    │ ○
14
    ○ ░
15
    ░    gitleaks
16

17
4:55PM INF 855 commits scanned.
18
4:55PM INF scan completed in 435ms
19
4:55PM WRN leaks found: 38

Gitleaks currently shows 38 leaks. You might be asking, how can a tool designed to find leaks have leaks itself? These are likely false positives, and the rules may need further tweaking to filter them out.

1
➜  gitleaks detect --source ./gitleaks -v
2
...
3
Finding:     ...e-cli login --token ******************************`
4
Secret:      ******************************
5
RuleID:      huggingface-access-token
6
Entropy:     4.553652
7
File:        cmd/generate/config/rules/huggingface.go
8
Line:        31
9
Commit:      9fb36b242d75aac1a2bf885724dfd9886db08ea7
10
Author:      *********
11
Email:       *********@users.noreply.github.com
12
Date:        2023-08-24T15:38:19Z
13
Fingerprint: 9fb36b242d75aac1a2bf885724dfd9886db08ea7:cmd/generate/config/rules/huggingface.go:huggingface-access-token:31
14
4:57PM INF 855 commits scanned.
15
4:57PM INF scan completed in 435ms
16
4:57PM WRN leaks found: 38

You can also use the -v (verbose) flag and get a full report.

Note: I replaced the actual secrets and personal details with asterisks ( * ) for privacy and security reasons.

What Gitleaks detected

The scan revealed multiple leaks, including API keys and tokens. For each finding, Gitleaks provided specific details:

1. File: The file containing the sensitive information.
2. Line Number: The exact line where the secret was found.
3. Commit Hash: The unique identifier for the commit that introduced the secret.
4. Author: The author of the commit.

These details are invaluable for understanding the context of the leak and taking corrective action. They help pinpoint when the leak occurred and who might be aware of it.

Scanning popular Github repositories with Gitleaks

Even well-maintained projects can have secrets accidentally committed. This demonstrates the importance of regularly scanning your repositories, regardless of the project’s size or reputation.

rust-lang/rust

1
➜  labs git clone [email protected]:rust-lang/rust.git
2
Cloning into 'rust'...
3
remote: Enumerating objects: 2672711, done.
4
remote: Counting objects: 100% (1168/1168), done.
5
remote: Compressing objects: 100% (661/661), done.
6
remote: Total 2672711 (delta 712), reused 776 (delta 497), pack-reused 2671543
7
Receiving objects: 100% (2672711/2672711), 1.26 GiB | 5.28 MiB/s, done.
8
Resolving deltas: 100% (2061551/2061551), done.
9
Updating files: 100% (48974/48974), done.
10

11
➜  labs gitleaks detect --source rust
12

13
    ○
14
    │╲
15
    │ ○
16
    ○ ░
17
    ░    gitleaks
18

19
5:43PM INF 185180 commits scanned.
20
5:43PM INF scan completed in 4m3s
21
5:43PM WRN leaks found: 2437
22
➜  labs

You read that right. Rust has 2,437 leaks, lol. Let’s be real here; the severity and legitimacy of some of them might vary. For example, this is reported as a security issue.

1
Finding:     ...ESS_KEY_ID: ${{ env.CACHES_AWS_ACCESS_KEY_ID }}
2
Secret:      CACHES_AWS_ACCESS_KEY_ID
3
RuleID:
4
Entropy:     3.188722
5
File:        .github/workflows/ci.yml
6
Line:        196
7
Commit:      1ca92c085788f68ff9b23cf597da5c62924e3f37
8
Author:      ************
9
Email:       ************
10
Date:        2024-04-29T19:32:35Z
11
Fingerprint: 1ca92c085788f68ff9b23cf597da5c62924e3f37:.github/workflows/ci.yml::196

We can’t see the actual value of CACHES_AWS_ACCESS_KEY_ID here. Even if we did, it wouldn’t pose a threat since cache access key IDs alone aren’t valuable to malicious users.

This shows why it’s crucial to double-check the results from any tool. Not every reported leak is a real security issue. For example:

• over 2000 “leaks” were DNA sequences formed entirely of the letters ACGT. The DNA sequences are in files like this one.
• 14 “leaks” were non-sensitive IDs from the AWS token pair, found in .travis.yaml
• 2 “leaks” were similar non-sensitive IDs in jobs.yml

It’s crucial to customize tools like Gitleaks to reduce false positives and focus on genuine threats. When projects are this complex, reporting large numbers of potential leaks without proper analysis can lead to unnecessary alarm and confusion.

Proper use of Gitleaks: A case study with the Rust repository

To effectively use Gitleaks and minimize false positives, it’s essential to tailor the tool to your project’s specific context.

Customizing gitleaks configuration

1. Create a .gitleaks.toml configuration file
2. Define rules and exceptions to refine the scan.

For example, if certain patterns (like DNA sequences or specific IDs) are not sensitive, they can be excluded.

1
[[rules]]
2
description = "Exclude DNA sequences"
3
regex = '''^[ACGT]{10,}$'''
4
tags = ["dna", "non-sensitive"]
5

6
[[rules]]
7
description = "Exclude AWS Access Key IDs"
8
regex = '''CACHES_AWS_ACCESS_KEY_ID'''
9
tags = ["aws", "non-sensitive"]

3. Run Gitleaks with the custom config: Use the —config flag to specify the custom configuration file.

1
gitleaks detect --source . --config .gitleaks.toml -v --report-path gitleaks-report.json

• —source ./rust: Specifies the directory of the Rust repository.
• —config .gitleaks.toml: Points to the custom configuration file.
• -v: Verbose mode for detailed output.
• —report-path gitleaks-report.json: Specifies the output file for the report.

Analyzing the Results

After running Gitleaks with a tailored configuration:

1
➜  rust git:(master) ✗ gitleaks detect --source . --config .gitleaks.toml
2

3

4
    ○
5
    │╲
6
    │ ○
7
    ○ ░
8
    ░    gitleaks
9

10
9:29PM INF 185180 commits scanned.
11
9:29PM INF scan completed in 3m59s
12
9:29PM WRN leaks found: 32
13
➜  rust git:(master)

The customized rules help eliminate common false positives, like non-sensitive IDs or known patterns. Feel free to tweak the rules and experimenting with the scanner too.

Preventing leaks with Gitleaks pre-commit hook

A pre-commit hook lets you run scripts before code is committed. Here’s a quick setup for Gitleaks as a pre-commit hook:

1. Install Gitleaks

1
brew install gitleaks

2. Create and Edit Hook Script:

1
touch your-project/.git/hooks/pre-commit

1
#!/bin/sh
2
gitleaks detect --source . -v --report-path gitleaks-report.json
3

4
if [ $? -ne 0 ]; then
5
  echo "Gitleaks detected leaks. Check gitleaks-report.json."
6
  exit 1
7
fi

• —source . Specifies the current directory as the source to scan. The . denotes the root of the current Git repository.

• -v: Stands for verbose mode, which provides more detailed output during the scan process. It helps in understanding what the tool is doing and any findings it may encounter.

• —report-path gitleaks-report.json: Specifies the path and filename where the scan results will be saved in JSON format. In this case, the report will be saved as gitleaks-report.json in the current directory.

3. Make Script Executable:

1
chmod +x .git/hooks/pre-commit

This setup scans the git changes for secrets before committing. If leaks are found, the commit is stopped, ensuring sensitive data stays out of your repository.

1
➜  labs git:(master) ✗ git add .
2
➜  labs git:(master) ✗ git commit -m "lezz goo"
3

4
    ○
5
    │╲
6
    │ ○
7
    ○ ░
8
    ░    gitleaks
9

10
10:31PM INF 2 commits scanned.
11
10:31PM INF scan completed in 4.02s
12
10:31PM INF no leaks found
13
[master 9c72b1ae] lezz goo
14
 1 file changed, 1 insertion(+), 1 deletion(-)
15
➜  labs git:(master) cat gitleaks-report.json
16
[]

How to evaluate tools for finding leaks

1. Open-Source Nature: The tool should have its source code available. This lets the community check for unauthorized data transmissions.

2. Network Monitoring: Monitor network traffic during scans. Ensure the tool doesn’t communicate with external servers.

Your code is only as safe as your secrets. Don’t let a tiny leak sink your ship.