jk VS pants

People have tried: https://github.com/jkcfg/jk

But yeah I agree. The thing is, if all you need is robust determinism why do you need a full functional language with currying and other complex concepts?

Google had the same problem for Bazel, and their solution (Starlark) is way easier to understand.

If I were writing a build system today (and I did just write one actually to test out some ideas) I would use Typescript for the language with something like jk to provide hermeticity. Typescript has many advantages, especially over Python, but mainly:

It's possible to sandbox most languages, and with some work you can probably make them deterministic too.

Here's an example: https://github.com/jkcfg/jk

That beats having to learn an entirely new language.

Maybe Javascript? A lot of web tools support Javascript config files. There's this nice-looking effort to provide a hermetic execution environment for them: https://github.com/jkcfg/jk and if you use Typescript you get an extremely good static type system too. Plus the language is already very well known with loads of tool support and documentation.

Definitely what I would use today.

If you think "but I need conditionals and file inclusion and ..." then maybe consider just allowing a full programming language instead. Someone pointed me to jk which looks like it is heading in the right direction, except that it outputs YAML by default for some insane reason.

You may be interested in jk. If you don't want to use a special purpose configuration language (jsonnet, cue, dhall, etc), this is a nice alternative that uses js in a hermetic runtime (but see their open issues for progress on that). They seem to also be adding native typescript support so you could even have type checking built-in.

> C/C++'s header system with conditional inclusion

Wouldn't it be more accurate to say something like "older build systems"? I don't think any of the things you listed are "modern". Which isn't a criticism of their legacy! They have been very useful for a long time, and that's to be applauded. But they have huge problems, which is a big part of why newer systems have been created.

FWIW, I have been using pants[0] (v2) for a little under a year. We chose it after also evaluating it and bazel (but not nix, for better or worse). I think it's really really great! Also painful in some ways (as is inevitably the case with any software). And of course it's nearly impossible to entirely stomp out "genrules" use cases. But it's much easier to get much closer to true hermeticity, and I'm a big fan of that.

0: https://www.pantsbuild.org/

Does pants/bazel can help me?

Hyper-large tech companies managing hyper-large monorepos using Bazel (google), buck (Facebook), please (thought machine), pants (Twitter, Foursquare & Square) enjoy them but also have a lot of resources devoted to running and maintaining it.

Yeah there's definitely some alternatives out there. Pants is another one that has a lot of traction.

You should look into pex, or it’s parent build system pants. A PEX (Python EXecutable) file can package up all your code including dependencies and run on another machine of similar OS with just an available compatible interpreter.

We don't have the CD yet, but I think what I put in place counts as simple CI (even if incomplete)? Every push & PR trigger an azure pipeline, which runs pants. This install the dependencies from the lockfile, run some linters, uses DVC to pull the data necessary for tests, and run unit tests (mypy check is deactivated until I solve a weird error). Basically the same script runs on laptops cross-platform (one of us uses Max, one Ubuntu with GPU, one Ubuntu with CPU, the scripts runs on every platform). The only difference with CI is the installation of Pants and the gestion of Cache (needs to be downloaded in CI so it takes ~3min in CI versus 20 seconds on my laptop).

This has also been recently integrated in pants.

https://github.com/pantsbuild/pants/issues/13393