trufflehog VS git-filter-repo

Trufflehog

Frankly, I think it will take years to replace API-keys (if it will ever happen). Developers are much better-off using CLI tools that prevent leaking secrets by blocking commits to git (e.g., https://github.com/Infisical/infisical or https://github.com/trufflesecurity/trufflehog)

Trufflehog also offers pre-commit hooks. You can have it report on PRs too.

Is this not just a another https://github.com/trufflesecurity/trufflehog?

I agree that code scanning is really important, the best way to convince others is to identify high-risk threats in source code and present them to the decision-makers. For example, scanning Secrets is great for showing how repositories can be a massive vulnerability and identifying some low-hanging fruit, especially in the git history. Attackers are really after git repository access for this reason and there are plenty of open-source or free tools that you can use to illustrate the problem. Git-Secrets, Truffle Hog. These aren't great for a long-term commercial solution, something like GitGuardian is a better commercial tool but if the goal is just to illustrate the problem then finding some high-value secrets with free tools is a good way to convince the security personnel to invest in some solutions. Then the door is open to having more conversations as you have already proven the risk.

It’s easy to think that it’s only important to scan for secrets in your public-facing repositories, but this real-world data breach proves that you need to treat all code the same from a security perspective. Malicious hackers can use open-source tools like Gitleaks and TruffleHog to quickly detect secrets in massive amounts of code*, without leaving a trace. As a defender, **it’s extremely important to have secret scans tightly integrated into your SDLC* (software development lifecycle) to reduce the risks of exposing them. GitGuardian offers secret scanning for private repositories in their Free, Business, and Enterprise plans.

There are software like Trufflehog ( https://github.com/trufflesecurity/trufflehog ), that finds secrets. We are using it at organizational level, but there's always some delay from finding something and getting it reported. I've been meaning to add it both to our CI so our team can notice right away, and even to Git push hooks, to catch these cases early.

https://github.com/trufflesecurity/trufflehog And https://github.com/Yelp/detect-secrets

Have you tried trufflehog or gitrob? gitrob trufflehog

**WARNING**: git-filter-branch has a glut of gotchas generating mangled history rewrites. Hit Ctrl-C before proceeding to abort, then use an alternative filtering tool such as 'git filter-repo' (https://github.com/newren/git-filter-repo/) instead. See the filter-branch manual page for more details; to squelch this warning, set FILTER_BRANCH_SQUELCH_WARNING=1. Proceeding with filter-branch... Rewrite a3a48b09e282854c80bf4ad02a017e249e161fd8 (2/8) (0 seconds passed, remaining 0 predicted) rm 'config.js' Rewrite 6e788e83a338e45b348d93d682b32c816ee2fbff (3/8) (0 seconds passed, remaining 0 predicted) rm 'config.js' Rewrite 7a378a0145bce70bea213ca5f9062138544db5f2 (4/8) (0 seconds passed, remaining 0 predicted) rm 'config.js' Rewrite 0637c9659623644cfceb35be10f2a1fe5c468e04 (5/8) (0 seconds passed, remaining 0 predicted) rm 'config.js' Rewrite 6c421eb99adc6b987cff7f3cada31e9313638072 (6/8) (0 seconds passed, remaining 0 predicted) rm 'config.js' Rewrite 98001e5b97270efa4a8ab5bd0452be56dd76883d (7/8) (0 seconds passed, remaining 0 predicted) rm 'config.js' Rewrite 2ca4e161a4af2b8f38c46faf848fdbb3e550f23c (8/8) (0 seconds passed, remaining 0 predicted) rm 'config.js' Ref 'refs/heads/secret_keys' was rewritten.

Yup you need https://github.com/newren/git-filter-repo Take a look at https://github.com/newren/git-filter-repo/blob/main/INSTALL.md for instructions

git filter-repo: a third-party command actually, as a replacement to Git's own filter-branch, that allows rewriting the whole history of a repository to remove a mistakenly added file, or help extract part of the repository to another.

I needed to archive some old repositories into a monorepo and of course I gave myself the requirement of maintaining git history, in some way. I tried a couple of solutions but it wasn't until I stumbled upon the git-filter-repo project at https://github.com/newren/git-filter-repo and another article which I've since lost (which was badly documented anyway) that I was able to figure out how to do this.

Git objects are not designed to be changed, they are immutable blobs. This is not a problem if you are making a reader, but is a problem when you want to change things, tools like old git-filter-branch or the newer filter repo abstract all reference updating away for you

Make sure your gitignore is setup right (GitHub has a repo of good defaults). If you messed that up, you could rewrite git history to remove the big stuff. Use git-filter-repo. Not sure how that works for LFS.

git-filter-repo

Check out git filter repo https://github.com/newren/git-filter-repo

I personally prefer git-bfg ( https://rtyley.github.io/bfg-repo-cleaner/ ) ... though git-filter-repo ( https://github.com/newren/git-filter-repo ) is quite popular. The difference for me was that git-bfg is JVM based and my work machine has Java on it while git-filter-repo is python based... and my work machine is without python.