resholve VS cue

I'm also not 100% on whether it answers the question, but I imagine you're thinking of https://github.com/abathur/resholve (doc in nixpkgs: https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/misc/resholve/README.md)

I intend to eventually find some time to figure out how feasible it would be to use https://github.com/abathur/resholve or wrapper techniques to bolt basalt (and perhaps other bash PMs) on to the nix ecosystem and nix-package some of your libraries.

(I'd love to have this ability for https://github.com/abathur/resholve to reliably identify arguments to one command that are also external commands/programs that it will in turn exec. I can't imagine trying to start it until/unless I have any bright ideas about how that executable spec and a parser for it would work.)

In https://github.com/abathur/resholve/blob/master/default.nix and https://github.com/abathur/resholve/blob/master/shell.nix you can see one approach to extending that line of thought to the default.nix itself.

Completions have in general been of interest, though the shell-specific completions I've looked at so far were all too dynamic.

I'd forgotten all about Fig since I saw your launch post here last year, so thanks for reminder. (I don't think I had quite started to work on parsing specific external commands, yet. Was still focused on just identifying the likely presence of exec in the executables.)

Are you familiar with the parse code? Are you handling painful stuff like combined short flags with a trailing option? (If I ferreted out some of the more painful cases I've had to wrangle, I am curious if you'd have a gut sense of whether your approach handles it. Would you mind if I reach out? I am working on this for https://github.com/abathur/resholve)

@dloreto @robrich A little aside from the announcement, but since it seems like you both work on this I wanted to surface something that came up down in a subthread:

I'm curious if you attempted to support macOS by doing this with Nix's dockerTools and cross-compiling (there may be better sources, but it's at least hinted at in https://nix.dev/tutorials/building-and-running-docker-images...)? If so, I'm wondering where that failed or bogged down?

---

Background: I build a tool (https://github.com/abathur/resholve) for ~packaging Bash/Shell (i.e., for demanding all dependencies be present). The tool's technically agnostic, but I built it specifically to fix Shell packaging in Nix.

I think it could benefit a lot of other Shell projects, since one of Shell's big tribulations is dealing with heterogenous environments, but most Shell projects wouldn't see much reason to endure the pain of adopting Nix if they still had to support the heterogenous environments.

Much like you're doing here, I've entertained figuring out how to build a Nix-based packaging flow that can generate deployable standalone bundles or containers. It'd be a heavy way to bundle Shell, but I imagine some projects would take the tradeoff for predictability and reduced support load. But since it would need to take place within a Nix build, I'd need to cross-compile for it to work on macOS. Hoping you know if it's a dead-end or not :)

For a concrete example, I've been developing a tool (https://github.com/abathur/resholve) that can ~build/link Bash/Shell scripts--i.e., rewrite them with external executables converted to absolute paths. (This helps ensure dependencies are known, declared, present, and don't have to be on the global PATH for the script to execute cleanly.)

There's a devilish sub-problem, which is that any given executable can potentially exec arbitrary arguments. For now I handle this with a very crude automated binary/executable analysis that needs to be augmented by human source analysis. Deep multi-language source analysis wouldn't be very scalable, but I suspect fairly-standardized structural annotations could improve the results in a scalable way.

I have to imagine there are other applications of the same information.

I'm not 100% sure if you see the minimal PATH dependencies as a problem or not--so this may or may not help--but I develop https://github.com/abathur/resholve to make it easier to package Shell in Nix/nixpkgs by rewriting invocations of external dependencies in Shell scripts to absolute paths--and shunit2 is one of the Nix packages that I've updated to use resholve.

I came here to say this, too :)

But, of course, it still isn't a silver bullet...

1. You still have to have a sane PATH. A fair amount of the Nix install-related issues that get opened are PATH problems, and you can also run into problems with PATH in cron/launchd.

2. You still have to know what the script depends on. This can get tricky beyond small scripts you wrote yourself. (I write a tool for ~linking/resolving external dependencies in Shell scripts, https://github.com/abathur/resholve. As I've been working on converting some of nixpkgs' existing Shell packages to use it, I almost always find dependencies the initial packager missed.)

Check out resholve. https://github.com/abathur/resholve

I'm continuing to work on https://concise-encoding.org which is a new security-conscious ad-hoc encoding format to replace JSON/XML and friends. I've been at it for 3 years so far and am close to a release.

In a nutshell:

- Edit in text, transmit in binary. One can be seamlessly converted to the other, but binary is far more efficient for processing, storage and transmission, while text is better for humans to read and edit (which happens far less often than the other things).

- Secure by design: Everything is tightly specced and accounted for so that there aren't differences between implementations that can be exploited to compromise your system. https://github.com/kstenerud/concise-encoding/blob/master/ce...

- Real type support because coercing everything into strings sucks (and is another security risk and source of incompatibilities).

XML had a good run but was replaced by JSON which was a big improvement. JSON also had a good run but it's time for it to retire now that the landscape has changed even further: Security and efficiency are the desires of today, and JSON provides neither.

I've got the spec nailed down and can finally see the light at the end of the tunnel for the reference implementation in golang. I still need to come up with a system for schemas, but I'm hoping that https://cuelang.org will fit the bill.

Has anyone taken a look at Cue who can share any experiences?

https://cuelang.org/

It's mentioned on the site as an alternative to Yaml. Recently watched (~half of) this intro to it: https://youtu.be/fR_yApIf6jU

the most interesting summary explanation of cue lang and its differences is from a bug filing - https://github.com/cuelang/cue/issues/33

>CUE is a bit different from the languages used in linguistics and more tailored to the general configuration issue as we've seen it at Google. But under the hood it adheres strictly to the concepts and principles of these approaches and we have been careful not to make the same mistakes made in BCL (which then were copied in all its offshoots). It also means that CUE can benefit from 30 years of research on this topic. For instance, under the hood, CUE uses a first-order unification algorithm, allowing us to build template extractors based on anti-unification (see issue #7 and #15), something that is not very meaningful or even possible with languages like BCL and Jsonnet.

I agree with you. Personally, I think Cue is much better than either YAML, TOML or JSON because it adds the concept of types to the idea of describing configuration.

true, but the tooling and workflow remains the same.

Not sure of any tool that could abstract the details sufficiently to be widely adopted. There is just too much nuance in cloud config.

I'm exploring using CUE (https://cuelang.org) to define TF resources, exporting as JSON for TF. So far it's much nicer