Pre-commit: framework for managing/maintaining multi-language pre-commit hooks

• pre-commit

  192 12,049 8.0 Python

  A framework for managing and maintaining multi-language pre-commit hooks.

  

You can point a hook towards a specific path, i.e. adding `files: ^C` to a hook will cause it to only run on the path of project/path C. There's also `exclude: ` to do to the opposite and other ways to customize each hook with granularity.

https://pre-commit.com/#pre-commit-configyaml---hooks

• Sentry

  266 36,886 10.0 Python

  Developer-first error tracking and performance monitoring

  

• InfluxDB

  www.influxdata.com sponsored

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• git-hooks.nix

  6 442 9.0 Nix

  Seamless integration of https://pre-commit.com git hooks with Nix.

  

> My least favourite bug is that it doesn't always play nicely with NixOS [1], and the maintainer locked me out of the issue for pointing it out.

Oof. Does this solve the problem https://github.com/cachix/pre-commit-hooks.nix (using Nix to manage dependencies)?

I was looking at pre-commit the other day, and wanted to incorporate it into the Nix setup of my projects.

• setup-dvc

  1 29 3.2 JavaScript

  DVC GitHub action

  

Here's our setup, which is the result of several iterations and ergonomics refinements. Note: our stack is 90% python, with TS for frontend. Also 95% devs use mac (there's one data scientist on windows, he uses WSL).

We install enough utilities with `brew` to get pyenv working, use that to build all python versions. Then iirc `brew install pipx`, maybe it's `pip3 install --user pipx`. Anyway, that's the only python library binary installed outside a venv.

Pipx installs isort, black, dvc, and pre-commit.

Every repo has a Makefile. This drives all the common operations. Pyproject.toml (/eslint.json?) set the config for isort and black (or eslint). `make format` runs isort and black on python, eslint on js. `make lint` just verifies.

Pre-commit only runs the lint, it doesn't format. It also runs some scripts to ensure you aren't accidentally committing large files. Pre-commit also runs several DVC actions (the default dvc hooks) on commit, push, and checkout. These run in a venv managed by pre-commit. We just pin the version.

Github actions has a dedicated lint.yaml which runs a python linter action. We use the black version here to define which black pipx installs. We use `act` if we wanna see how an action runs without sending a commit just to trigger jobs.

As an aside, I'm still fiddling with the dvc `pre-commit` post-checkout hooks. They don't always pull the files when they ought to.

Most of the actual unit/integration tests run in containers, but they can run in a venv with the same logic, thanks to makefile. We use a dvc action to sync files in CI.

So yeah there's technically 2 copies of black and dvc, but we just use pinning. In practice, we've only had one issue with discrepancies in behavior locally vs CI, which was local black not catching a rule to avoid ''' for docstrings; using """ fixed it. On the whole, pre-commit saves against a lot of annoying goofs, but CI system is law, so we largely harmonize against that.

IMHO, this is the least egregious "double accounting" we have in local vs staging ci vs production ci (I lost that battle, manager would rather keep staing.yaml and production.yaml, rather than parameterize. Shrug.gif).

Technologies referenced:

https://dvc.org/

https://github.com/iterative/setup-dvc

https://github.com/marketplace/actions/python-linter

https://github.com/nektos/act

• dvc

  109 13,116 9.7 Python

  🦉 ML Experiments and Data Management with Git

  

Here's our setup, which is the result of several iterations and ergonomics refinements. Note: our stack is 90% python, with TS for frontend. Also 95% devs use mac (there's one data scientist on windows, he uses WSL).

We install enough utilities with `brew` to get pyenv working, use that to build all python versions. Then iirc `brew install pipx`, maybe it's `pip3 install --user pipx`. Anyway, that's the only python library binary installed outside a venv.

Pipx installs isort, black, dvc, and pre-commit.

Every repo has a Makefile. This drives all the common operations. Pyproject.toml (/eslint.json?) set the config for isort and black (or eslint). `make format` runs isort and black on python, eslint on js. `make lint` just verifies.

Pre-commit only runs the lint, it doesn't format. It also runs some scripts to ensure you aren't accidentally committing large files. Pre-commit also runs several DVC actions (the default dvc hooks) on commit, push, and checkout. These run in a venv managed by pre-commit. We just pin the version.

Github actions has a dedicated lint.yaml which runs a python linter action. We use the black version here to define which black pipx installs. We use `act` if we wanna see how an action runs without sending a commit just to trigger jobs.

As an aside, I'm still fiddling with the dvc `pre-commit` post-checkout hooks. They don't always pull the files when they ought to.

Most of the actual unit/integration tests run in containers, but they can run in a venv with the same logic, thanks to makefile. We use a dvc action to sync files in CI.

So yeah there's technically 2 copies of black and dvc, but we just use pinning. In practice, we've only had one issue with discrepancies in behavior locally vs CI, which was local black not catching a rule to avoid ''' for docstrings; using """ fixed it. On the whole, pre-commit saves against a lot of annoying goofs, but CI system is law, so we largely harmonize against that.

IMHO, this is the least egregious "double accounting" we have in local vs staging ci vs production ci (I lost that battle, manager would rather keep staing.yaml and production.yaml, rather than parameterize. Shrug.gif).

Technologies referenced:

https://dvc.org/

https://github.com/iterative/setup-dvc

https://github.com/marketplace/actions/python-linter

https://github.com/nektos/act

• pip-audit

  22 915 8.8 Python

  Audits Python environments, requirements files and dependency trees for known security vulnerabilities, and can automatically fix them

  

This is why I stuff everything into a top-level Makefile with `.PHONY` rules instead. Nearly every developer knows how to invoke `make` and already has tab completion for `make` rules, to boot.

For example: https://github.com/trailofbits/pip-audit/blob/main/Makefile

• lint-staged

  50 12,858 8.4 JavaScript

  🚫💩 — Run linters on git staged files

  

Black is an interesting example because you want two different commands in your pre-commit versus CI/CD, at least in my estimation.

In pre-commit you want to run Black in full formatter mode on the staged files. (VS Code's "Format After Save" does most of the work already, but every now and then it slips on something.) In CI/CD you want the check that formatting occurred and block it as a failed test. Those are different commands.

For what it is worth, I was evaluating lint-staged [1] and husky [2] for my pre-commit hooks.

[1] https://github.com/okonet/lint-staged

[2] https://github.com/typicode/husky

• WorkOS

  workos.com sponsored

  The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

  

• husky

  125 31,546 8.8 JavaScript

  Git hooks made easy 🐶 woof!

Black is an interesting example because you want two different commands in your pre-commit versus CI/CD, at least in my estimation.

In pre-commit you want to run Black in full formatter mode on the staged files. (VS Code's "Format After Save" does most of the work already, but every now and then it slips on something.) In CI/CD you want the check that formatting occurred and block it as a failed test. Those are different commands.

For what it is worth, I was evaluating lint-staged [1] and husky [2] for my pre-commit hooks.

[1] https://github.com/okonet/lint-staged

[2] https://github.com/typicode/husky

Suggest a related project