Add secret-scan: git history secret sweep tooling - #107
Draft
longshuicy wants to merge 1 commit into
Draft
longshuicy wants to merge 1 commit into
longshuicy wants to merge 1 commit into
Conversation
Adds a script that sweeps the full git history of every IN-CORE repo with gitleaks and trufflehog and compiles a redacted report. Covers the two gaps in GitHub's own secret scanning: private repos (not available on the org's free plan, currently 9 of 29 repos) and credentials that don't match GitHub's ~200 partner patterns — DB passwords, Keycloak client secrets, connection strings, committed .pem files. Scans mirror clones so commits reachable only from deleted branches are included. TruffleHog can't read bare repos, so each is materialised as a temporary --shared working clone; objects are shared via alternates, and refs from every branch stay visible. Live credential verification is opt-in via VERIFY=1 and off by default, since it exercises any real key found against the provider's API. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of the history secret sweep in #48.
Adds a
secret-scan/folder with the script used for the one-time sweep across the org, plus a README.Why
GitHub's secret scanning is enabled on the public repos and it does backfill full history when you turn it on — that coverage is real. But it leaves two gaps:
.pem— which is most of our realistic exposure.So this runs gitleaks + trufflehog over full history to cover the rest.
Notes for reviewers
.gitand an index). Each mirror is materialised as a temporarygit clone --sharedworking clone; objects are shared via alternates so history isn't duplicated, and I verified a secret planted on a non-checked-out branch is still found.VERIFY=1), off by default. It's what makes results triageable, but it exercises any real key found against AWS/GitHub/Slack/etc. from the machine running the scan, where live hits may surface in the provider's audit log. That felt like it should be a deliberate choice rather than the default..gitignoreexcludessweep/andreports/— raw scan output contains unredacted secrets. The compiledreport.mdredacts to first/last four chars so it's safe to paste into an issue.Status of the actual sweep
Draft because the triage isn't finished. Current raw counts across 29 repos: 88 gitleaks, 378 trufflehog — both pre-triage and expected to be mostly UUIDs, hashes, and lockfile digests. The 9 private repos, which had no coverage at all, came back clean apart from two DataWolf parameter UUIDs that are false positives.
I'll post the triaged results in #48.
🤖 Generated with Claude Code