git-internals-pdf VS git-filter-repo

For me the Peepcode Git Internals book was a great peek under the hood. I went from "Git has a lot of magical incantations" to "Git is pretty simple and I could probably build a version of it".

I generally recommend the Peepcode Git Internals book. The first half explains the internals of how Git works, and the second half is a command reference.

Once you get the hang of basic Git operations, you should look into how Git works under the hood. Git Internals helped me a lot on this.

❯ git log --graph --pretty=oneline --abbrev-commit --all * 150c57d (HEAD -> squash-merge) Squashed commit of the following: | * 535b740 (no-squash-merge) Merge branch 'work-branch' into no-squash-merge |/| | * 1836f1c (work-branch) And more | * 4b84cfe Add more |/ * 16660f8 (main) Add more * 02a154b Initial commit ❯ git cat-file -p no-squash-merge tree 58c1fb22faa444b264e98a5ae4c4ddb07be09697 parent 16660f8b1d1538ed1b55d8533b3ee7feb68e474c parent 1836f1c53221ae701a038bf5ae380770ea911665 author Manuel Odendahl 1653304391 -0400 committer Manuel Odendahl 1653304391 -0400 Merge branch 'work-branch' into no-squash-merge * work-branch: And more Add more squash-merges-considered-harmful on  squash-merge on ☁️ ttc (us-east-1) ❯ git cat-file -p squash-merge tree 58c1fb22faa444b264e98a5ae4c4ddb07be09697 parent 16660f8b1d1538ed1b55d8533b3ee7feb68e474c author Manuel Odendahl 1653304543 -0400 committer Manuel Odendahl 1653304543 -0400 Squashed commit of the following: commit 1836f1c53221ae701a038bf5ae380770ea911665 Author: Manuel Odendahl Date: Mon May 23 07:11:08 2022 -0400 And more commit 4b84cfe11aa51da994448e602e1bc4cc6083d691 Author: Manuel Odendahl Date: Mon May 23 07:11:03 2022 -0400 Add more * ``` {% endraw %} You can see that save that both {% raw %}`squash-merge`{% endraw %} and {% raw %}`no-squash-merge`{% endraw %} point to the exact same tree. The only changed thing is the commit message, and the missing parent in the squash merge. To read more about the underpinnings of git, I can recommend just experimenting with the git command line, and the following resources: - [Building Git by James Coglan](https://shop.jcoglan.com/building-git/) - [Git Internals by Scott Chacon](https://github.com/pluralsight/git-internals-pdf) ## But the history! But Manuel, you say, the history is so much cleaner! To which I counter that it is actually not. If you want to hide the link to the right parent of the non-squash merge (as it is called, the left parent being {% raw %}`main`{% endraw %} ), all you need to do is to hide it. If you use the command-line or a proper tool, use the option to only show first parents. If you only look at the first parent, and configure your git tool to fill in a full log history of the branch into the merge commit message (I personally use the github CLI {% raw %}`gh`{% endraw %} or some git-commit hooks to do it), the squash merge commit is identical to the non squash merge commit. A favorite {% raw %}`git log`{% endraw %} command of mine to quickly look at the history of the main branch, and create a changelog: {% raw %} ```shell > git log --pretty=format:'# %ad %H %s' --date=short --first-parent --reverse # 2022-05-23 02a154bc4f0fa9bca567676d45d136619c076a95 Initial commit # 2022-05-23 16660f8b1d1538ed1b55d8533b3ee7feb68e474c Add more # 2022-05-23 535b740f42e331175f3766c1374116e329a78f7e Merge branch 'work-branch' into no-squash-merge

I often recommend the Peepcode Git Internals book. The first half talks about how Git works internally. The second half is a "how to use Git" tutorial. I think understanding the internals (which aren't really that complicated) can really help to demystify Git.

Conceptually, Git's not really all that complicated. I always recommend people to read the Peepcode Git Internals book (originally $9, now free): https://github.com/pluralsight/git-internals-pdf/blob/master/drafts/peepcode-git.pdf

Check out this page if you haven't yet (alternatively, direct link to the PDF.) I hear it's all good, but the Understanding Git chapter is the one I'd specifically point you to.

I still very much recommend the Peepcode Git Internals book.

https://github.com/pluralsight/git-internals-pdf/releases

**WARNING**: git-filter-branch has a glut of gotchas generating mangled history rewrites. Hit Ctrl-C before proceeding to abort, then use an alternative filtering tool such as 'git filter-repo' (https://github.com/newren/git-filter-repo/) instead. See the filter-branch manual page for more details; to squelch this warning, set FILTER_BRANCH_SQUELCH_WARNING=1. Proceeding with filter-branch... Rewrite a3a48b09e282854c80bf4ad02a017e249e161fd8 (2/8) (0 seconds passed, remaining 0 predicted) rm 'config.js' Rewrite 6e788e83a338e45b348d93d682b32c816ee2fbff (3/8) (0 seconds passed, remaining 0 predicted) rm 'config.js' Rewrite 7a378a0145bce70bea213ca5f9062138544db5f2 (4/8) (0 seconds passed, remaining 0 predicted) rm 'config.js' Rewrite 0637c9659623644cfceb35be10f2a1fe5c468e04 (5/8) (0 seconds passed, remaining 0 predicted) rm 'config.js' Rewrite 6c421eb99adc6b987cff7f3cada31e9313638072 (6/8) (0 seconds passed, remaining 0 predicted) rm 'config.js' Rewrite 98001e5b97270efa4a8ab5bd0452be56dd76883d (7/8) (0 seconds passed, remaining 0 predicted) rm 'config.js' Rewrite 2ca4e161a4af2b8f38c46faf848fdbb3e550f23c (8/8) (0 seconds passed, remaining 0 predicted) rm 'config.js' Ref 'refs/heads/secret_keys' was rewritten.

Yup you need https://github.com/newren/git-filter-repo Take a look at https://github.com/newren/git-filter-repo/blob/main/INSTALL.md for instructions

git filter-repo: a third-party command actually, as a replacement to Git's own filter-branch, that allows rewriting the whole history of a repository to remove a mistakenly added file, or help extract part of the repository to another.

I needed to archive some old repositories into a monorepo and of course I gave myself the requirement of maintaining git history, in some way. I tried a couple of solutions but it wasn't until I stumbled upon the git-filter-repo project at https://github.com/newren/git-filter-repo and another article which I've since lost (which was badly documented anyway) that I was able to figure out how to do this.

Git objects are not designed to be changed, they are immutable blobs. This is not a problem if you are making a reader, but is a problem when you want to change things, tools like old git-filter-branch or the newer filter repo abstract all reference updating away for you

Make sure your gitignore is setup right (GitHub has a repo of good defaults). If you messed that up, you could rewrite git history to remove the big stuff. Use git-filter-repo. Not sure how that works for LFS.

git-filter-repo

Check out git filter repo https://github.com/newren/git-filter-repo

I personally prefer git-bfg ( https://rtyley.github.io/bfg-repo-cleaner/ ) ... though git-filter-repo ( https://github.com/newren/git-filter-repo ) is quite popular. The difference for me was that git-bfg is JVM based and my work machine has Java on it while git-filter-repo is python based... and my work machine is without python.