The Python Package Index is now a GitHub secret scanning integrator

• caulking

  2 27 4.4 Shell

  Prevent leaks with gitleaks, and use tests to validate

  

Learning so much from this thread. If used these tools when i knew what to look for but thats been the tricky bit.

psanford also mentioned truffleHog and others, lstamour mentioned https://github.com/cloud-gov/caulking which is built on gitleaks which like a lot. caulking's customized list of patterns for gitleaks is hee https://github.com/cloud-gov/caulking/blob/master/local.toml Looks like it would have found the keys in my example case no problem.

• gh-action-pypi-publish

  5 834 8.3 Python

  The blessed :octocat: GitHub Action, for publishing your :package: distribution files to PyPI: https://github.com/marketplace/actions/pypi-publish

  

• WorkOS

  workos.com sponsored

  The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

  

• trufflehog

  25 13,863 9.9 Go

  Find and verify credentials

  

There are tools available to help look for this sort of thing (for both you and any potential attackers). TruffleHog[1] is the first one that comes to mind for me.

I also like shhgit[2] for looking for secrets in repositories. (I don't think shhgit will look back in the git history for you though).

[1]: https://github.com/dxa4481/truffleHog

• roadmap

  66 7,688 0.0

  GitHub public roadmap

  

It's on their public roadmap: https://github.com/github/roadmap/issues/94

Unfortunately it's marked as "Future," so it's still a ways out.

• Zulip

  117 20,023 10.0 Python

  Zulip server and web application. Open-source team chat that helps teams stay productive and focused.

  

When I helped to take Zulip open-source in 2015, I wrote a simple script that scrubbed secrets from the commit history using git fast-export and git fast-import. We replaced all our secrets with xxxxxxx placeholders, replaced internal customer references with dummy names, deleted and renamed certain files, and even did some code replacements that caused certain commit diffs to become empty so we could remove those commits from the history.

https://github.com/zulip/zulip/blob/3.3/tools/zanitizer

https://github.com/zulip/zulip/blob/3.3/tools/zanitizer_conf...

The script was really fast (all ~10000 commits in a few minutes), which allowed us to iterate quickly on its configuration as we audited using gitk and other tools for remaining items to scrub.

Doing this work allowed us to release with an essentially complete history going back to the first commit in 2012, which has been a really valuable resource for understanding why various Zulip subsystems were written the way they were.

Nowadays there are other tools for scrubbing history that might be more polished, like BFG: https://rtyley.github.io/bfg-repo-cleaner/

• warehouse

  274 3,465 9.7 Python

  The Python Package Index

  

In case anyone is interested, it looks like this is the implementation on the PyPI side: https://github.com/pypa/warehouse/pull/8563

• git-filter-repo

  50 7,372 1.0 Python

  Quickly rewrite git repository history (filter-branch replacement)

[1] https://github.com/newren/git-filter-repo

• InfluxDB

  www.influxdata.com sponsored

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

Suggest a related project