Git-annex – Managing large files with Git

• dud

  14 166 6.3 Go

  A lightweight CLI tool for versioning data alongside source code and building data pipelines.

Thanks for sharing your experience. It's non-trivial and surprising behavior like this that drove me to build a custom system[0] myself. When I started researching version control tools for large files, I remember feeling like git-annex and Git LFS were awkwardly bolted onto Git; Git simply wasn't designed for large files. Then I found DVC[1], and its approach rang true for me. However, after using DVC for a year or so, I grew tired of DVC's many puzzling behaviors (most of which are outlined in the README at [0]). In the end, I built the tool I wanted for the job -- one that is exceptionally simple and fast.

[0]: https://github.com/kevin-hanselman/dud

• dvc

  109 13,116 9.7 Python

  🦉 ML Experiments and Data Management with Git

  

• WorkOS

  workos.com sponsored

  The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

  

Suggest a related project