git-subrepo VS git-filter-repo

git-subrepo - complicated and difficult to understand

I find the following approach more consistent to manage components: https://github.com/ingydotnet/git-subrepo . And native package management systems, like npm in JavaScript universe, superior to either of the above. But the choice of a particular method depends on problems we need to solve. In terms of one-time codebase aggregation method they are all equally fine.

I much prefer https://github.com/ingydotnet/git-subrepo

Where I work we just package everything (nugets, python packages, npm) on our Artifactory. Contracts dependencies (DLLs, protobufs) are also distributed as packages. We made it easy to fetch and test the source and allow developers to develop, debug and test those dependencies with their own project if needed.

Every time we try to assemble repositories in macro-repos we always end up regretting it. Multiple dedicated repositories allow autonomy for teams and enforce modularity and coding as a library. Monorepos have a tendency of becoming huge merge trains easily and often derailed and with lots of fear of being blamed on stepping on someone else's toes.

We update often all our projects knowing full well that not doing so is just borrowing development time at high interest rate.

As a side-note when we do have to do an assembly of different code base, we use git-subrepo: https://github.com/ingydotnet/git-subrepo which provide the best of both submodules and subtree.

v2: I use git subrepo or a similar tool, to get the upstream yaml into my repo.

I found git-subrepo: https://github.com/ingydotnet/git-subrepo

You might also check out git subrepo.

Besides these git x-modules, there are historically three contenders:

git submodules: https://git-scm.com/book/en/v2/Git-Tools-Submodules

git subtrees: https://www.atlassian.com/git/tutorials/git-subtree

git subrepos: https://github.com/ingydotnet/git-subrepo

---

git subrepos work simply by copying your dependency to a subdirectory and committing the changes using one large commit that retains metadata about the update to the subrepo. For that reason, git subrepos aren't symlinks. You don't need to git clone --recursive like with git submodules, and you don't need cross-repo authentication. Updating a subrepo means performing another commit.

Even though git subrepos are the most poorly maintained, the design is simpler.

I wish someone would fork and take over maintenance.

git subrepos are the best.

**WARNING**: git-filter-branch has a glut of gotchas generating mangled history rewrites. Hit Ctrl-C before proceeding to abort, then use an alternative filtering tool such as 'git filter-repo' (https://github.com/newren/git-filter-repo/) instead. See the filter-branch manual page for more details; to squelch this warning, set FILTER_BRANCH_SQUELCH_WARNING=1. Proceeding with filter-branch... Rewrite a3a48b09e282854c80bf4ad02a017e249e161fd8 (2/8) (0 seconds passed, remaining 0 predicted) rm 'config.js' Rewrite 6e788e83a338e45b348d93d682b32c816ee2fbff (3/8) (0 seconds passed, remaining 0 predicted) rm 'config.js' Rewrite 7a378a0145bce70bea213ca5f9062138544db5f2 (4/8) (0 seconds passed, remaining 0 predicted) rm 'config.js' Rewrite 0637c9659623644cfceb35be10f2a1fe5c468e04 (5/8) (0 seconds passed, remaining 0 predicted) rm 'config.js' Rewrite 6c421eb99adc6b987cff7f3cada31e9313638072 (6/8) (0 seconds passed, remaining 0 predicted) rm 'config.js' Rewrite 98001e5b97270efa4a8ab5bd0452be56dd76883d (7/8) (0 seconds passed, remaining 0 predicted) rm 'config.js' Rewrite 2ca4e161a4af2b8f38c46faf848fdbb3e550f23c (8/8) (0 seconds passed, remaining 0 predicted) rm 'config.js' Ref 'refs/heads/secret_keys' was rewritten.

Yup you need https://github.com/newren/git-filter-repo Take a look at https://github.com/newren/git-filter-repo/blob/main/INSTALL.md for instructions

git filter-repo: a third-party command actually, as a replacement to Git's own filter-branch, that allows rewriting the whole history of a repository to remove a mistakenly added file, or help extract part of the repository to another.

I needed to archive some old repositories into a monorepo and of course I gave myself the requirement of maintaining git history, in some way. I tried a couple of solutions but it wasn't until I stumbled upon the git-filter-repo project at https://github.com/newren/git-filter-repo and another article which I've since lost (which was badly documented anyway) that I was able to figure out how to do this.

Git objects are not designed to be changed, they are immutable blobs. This is not a problem if you are making a reader, but is a problem when you want to change things, tools like old git-filter-branch or the newer filter repo abstract all reference updating away for you

Make sure your gitignore is setup right (GitHub has a repo of good defaults). If you messed that up, you could rewrite git history to remove the big stuff. Use git-filter-repo. Not sure how that works for LFS.

git-filter-repo

Check out git filter repo https://github.com/newren/git-filter-repo

I personally prefer git-bfg ( https://rtyley.github.io/bfg-repo-cleaner/ ) ... though git-filter-repo ( https://github.com/newren/git-filter-repo ) is quite popular. The difference for me was that git-bfg is JVM based and my work machine has Java on it while git-filter-repo is python based... and my work machine is without python.