resholve VS Git

I'm also not 100% on whether it answers the question, but I imagine you're thinking of https://github.com/abathur/resholve (doc in nixpkgs: https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/misc/resholve/README.md)

I intend to eventually find some time to figure out how feasible it would be to use https://github.com/abathur/resholve or wrapper techniques to bolt basalt (and perhaps other bash PMs) on to the nix ecosystem and nix-package some of your libraries.

(I'd love to have this ability for https://github.com/abathur/resholve to reliably identify arguments to one command that are also external commands/programs that it will in turn exec. I can't imagine trying to start it until/unless I have any bright ideas about how that executable spec and a parser for it would work.)

In https://github.com/abathur/resholve/blob/master/default.nix and https://github.com/abathur/resholve/blob/master/shell.nix you can see one approach to extending that line of thought to the default.nix itself.

Completions have in general been of interest, though the shell-specific completions I've looked at so far were all too dynamic.

I'd forgotten all about Fig since I saw your launch post here last year, so thanks for reminder. (I don't think I had quite started to work on parsing specific external commands, yet. Was still focused on just identifying the likely presence of exec in the executables.)

Are you familiar with the parse code? Are you handling painful stuff like combined short flags with a trailing option? (If I ferreted out some of the more painful cases I've had to wrangle, I am curious if you'd have a gut sense of whether your approach handles it. Would you mind if I reach out? I am working on this for https://github.com/abathur/resholve)

@dloreto @robrich A little aside from the announcement, but since it seems like you both work on this I wanted to surface something that came up down in a subthread:

I'm curious if you attempted to support macOS by doing this with Nix's dockerTools and cross-compiling (there may be better sources, but it's at least hinted at in https://nix.dev/tutorials/building-and-running-docker-images...)? If so, I'm wondering where that failed or bogged down?

---

Background: I build a tool (https://github.com/abathur/resholve) for ~packaging Bash/Shell (i.e., for demanding all dependencies be present). The tool's technically agnostic, but I built it specifically to fix Shell packaging in Nix.

I think it could benefit a lot of other Shell projects, since one of Shell's big tribulations is dealing with heterogenous environments, but most Shell projects wouldn't see much reason to endure the pain of adopting Nix if they still had to support the heterogenous environments.

Much like you're doing here, I've entertained figuring out how to build a Nix-based packaging flow that can generate deployable standalone bundles or containers. It'd be a heavy way to bundle Shell, but I imagine some projects would take the tradeoff for predictability and reduced support load. But since it would need to take place within a Nix build, I'd need to cross-compile for it to work on macOS. Hoping you know if it's a dead-end or not :)

For a concrete example, I've been developing a tool (https://github.com/abathur/resholve) that can ~build/link Bash/Shell scripts--i.e., rewrite them with external executables converted to absolute paths. (This helps ensure dependencies are known, declared, present, and don't have to be on the global PATH for the script to execute cleanly.)

There's a devilish sub-problem, which is that any given executable can potentially exec arbitrary arguments. For now I handle this with a very crude automated binary/executable analysis that needs to be augmented by human source analysis. Deep multi-language source analysis wouldn't be very scalable, but I suspect fairly-standardized structural annotations could improve the results in a scalable way.

I have to imagine there are other applications of the same information.

I'm not 100% sure if you see the minimal PATH dependencies as a problem or not--so this may or may not help--but I develop https://github.com/abathur/resholve to make it easier to package Shell in Nix/nixpkgs by rewriting invocations of external dependencies in Shell scripts to absolute paths--and shunit2 is one of the Nix packages that I've updated to use resholve.

I came here to say this, too :)

But, of course, it still isn't a silver bullet...

1. You still have to have a sane PATH. A fair amount of the Nix install-related issues that get opened are PATH problems, and you can also run into problems with PATH in cron/launchd.

2. You still have to know what the script depends on. This can get tricky beyond small scripts you wrote yourself. (I write a tool for ~linking/resolving external dependencies in Shell scripts, https://github.com/abathur/resholve. As I've been working on converting some of nixpkgs' existing Shell packages to use it, I almost always find dependencies the initial packager missed.)

Check out resholve. https://github.com/abathur/resholve

Look at any PR/patch series that got merged into the Git project. https://github.com/git/git/

Any random one. Because those that did not meet the minimum criteria for a well-crafted history would not have passed review.

Boy, I can't find this either (but also, the kernel mailing list is _really_ difficult to search). I really remember Linus saying something like "it's not a real SCM, but maybe someone could build one on top of it someday" or something like that, but I cannot figure out how to find that.

You _can_ see, though, that in his first README, he refers to what he's building as not a "real SCM":

https://github.com/git/git/commit/e83c5163316f89bfbde7d9ab23...

Here is the direct link, as HN somehow removes the query string: https://github.com/git/git/commits?author=peff&since=2023-10...

I understand all that.

I'm saying, if you write a survey and one of the possible answers is "diff", but you don't clearly define what you mean by "diff", then don't be surprised if respondents use any reasonable definition that makes sense to them. Ask an ambiguous question, get a mishmash of answers.

The thing that Git uses for packfiles is called a "delta" by Git, but it's also reasonable to call it a "diff". After all, Git's delta algorithm is "greatly inspired by parts of LibXDiff from Davide Libenzi"[1]. Not LibXDelta but LibXDiff.

Yes, how Git stores blobs (using deltas) is orthogonal to how Git uses blobs. But while that orthogonality is useful for reasoning about Git, it's not wrong to think of a commit as the totality of what Git does, including that optimization. (Some people, when learning Git, stumble over the way it's described as storing full copies, think it's wasteful. For them to wrap their heads around Git, they have to understand that the optimization exists. Which makes sense because Git probably wouldn't be practical if it lacked that optimization.)

The reason I'm bringing all this up is, if you're trying to explain Git, which is what the original article is about, then it's very important to keep in mind that someone who is learning Git needs to know what you mean when you say "diff". Most people who already know Git would tend to gravitate toward the definition of "diff" that you're assuming (the thing that Git computes on the fly and never stores), but people who already know Git aren't the target audience when you're teaching Git.

---

[1] https://github.com/git/git/blob/master/diff-delta.c