jsonmerge_git_merge_driver VS locust

I investigated the option of using a custom git merge driver for a project where we were planning to version control a bunch of data files using git.

Here's a proof of concept python merge driver I bashed together at the time to auto-merge JSON objects: https://github.com/fcostin/jsonmerge_git_merge_driver

This never went anywhere near production, but it was very easy to put together something basic.

One complication with using a custom merge driver, as discussed by https://github.com/Praqma/git-merge-driver , is that they need to be configured inside the `.git/config` of the repo, which itself is not version controlled. So there's an additional config management overhead to rolling that out to everyone in a machine. Additionally, if outsourcing hosting for git repos, it may not be supported to install and configure a custom merge driver for merges conducted by the hosting platform (e.g. merges created by github.com pull request workflow).

One idea I had at the time was using external schema files (e.g. JSON schema for JSON files) to help guide/constrain the result of the merge. I never implemented it, but it should be possible. If the schemas were also version controlled in the same git repo that stores the data, you'd need to figure out which one (and which version) to load when resolving a merge conflict of a data file. There doesn't seem to be a well-supported robust way for a merge driver script to discover the source and destination branches, but there are some potentially fragile ways of doing it that work some of the time.

Nice!

Have been working on something similar, although my use case is more about learning how code has changed across git commits: https://github.com/bugout-dev/locust

For Javascript/Typescript/React support, like you, I hooked into the Babel toolchain. Can't recommend it highly enough.

There's also a newish project called quick-lint-js which seems to have written their own from-scratch AST parser for JS, but I haven't tried it yet: https://github.com/quick-lint/quick-lint-js

Finally, another project that I know in this space is comby (I believe it is owned/maintained by the folks at Sourcegraph): https://comby.dev/

Don't know why I dumped all those links there. Just figured there may be something useful in them for you. Am also just super passionate about building knowledge about code bases by analyzing their ASTs. Nice to meet a fellow enthusiast. :)

I maintain a free/open source project that does exactly what the author asks for: https://github.com/bugout-dev/locust.

Our tool uses git as the foundation of its functionality. It superimposes git diffs on top of ASTs.

It is insanely powerful.

For example, we use it to power semantic code search and current support Python, Javascript, and Java. We generate a JSON object defining the AST differences between initial and terminal commits on GitHub PRs and doing text search on the JSON objects performs surprisingly well when we want to answer questions like, "When did we add dateutils as a dependency?" or "When did we last change the /journals handler on the API?"

The Python integration currently sees the most use but if you are interested in other languages, we would be happy to support it.

Do drop me a DM if you want help getting started with Locust.

My team has a similar project (Locust: https://github.com/bugout-dev/locust) where the goal is to learn the semantic meanings of code changes in git commits, GitHub PRs, etc.

Since we took git diffs as a target for semantic analysis, we have a different approach to our diffs. We start with line-by-line diffs (specifically using "git diff") and then take a semantic diff by superimposing the git diff information on top of the initial and terminal ASTs.

This makes the diff calculation cheaper because we don't have to do full diff between trees.

Haven't updated the code in a few months, but my team is actively using Locust on public GitHub repos to learn the semantics of those code bases. We do plan to do some work on it soon to make it easier to make Locust easier to use (especially as a library).

Really need to sit down and take a proper look at tree-sitter. We currently support Locust diffs for Python, Javascript, and Java, but each one is custom written and implements the same basic algorithm. It looks like tree sitter might just crush this problem for us.