Modern CI Is Too Complex and Misdirected

This page summarizes the projects mentioned and recommended in the original post on news.ycombinator.com

Our great sponsors
  • WorkOS - The modern identity platform for B2B SaaS
  • InfluxDB - Power Real-Time Data Analytics at Scale
  • SaaSHub - Software Alternatives and Reviews
  • blob

    > Bazel has remote execution and remote caching as built-in features. Hey - those are built-in features of modern CI systems too! So here's a thought experiment: if I define a build system in Bazel and then define a server-side Git push hook so the remote server triggers Bazel to build, run tests, and post the results somewhere, is that a CI system? I think it is! A crude one. But I think that qualifies as a CI system.

    Bazel can be a pain. Integration with external package managers is unnatural, hermetic toolchains are tricky, naively-written rules end up using system-provided utilities, breaking reproducibility.

    But the remote execution API (besides feeling like magic) lets you treat your CI as "just another user" by design, unning the same `bazel test //...` or whatever else. This means

    - the CI DSL/YAML files tend to have mostly publishing and other CI-specific information in then (this feels right)

    - you get to debug your build pipeline locally

    - you are actually testing the ability of a new user to pull the repo, build, and have everything just work. There is no special CI build environment that users implicit have to match.

    - tangentially: the remote execution API is beautiful in its simplicity:

    https://github.com/bazelbuild-remote-apis/blob/master/build/...

  • garden

    Automation for Kubernetes development and testing. Spin up production-like environments for development, testing, and CI on demand. Use the same configuration and workflows at every step of the process. Speed up your builds and test runs via shared result caching

    I cannot stop thinking this guy is describing what we do at https://garden.io.

    He seems to go on describing the Stack Graph and the build/test/deploy/task primitives, the unified configuration between environments, build and test results caching, the platform agnosticism (even though we are heavy focused on kubernetes) and the fact that CI can be just a feature, not a product on itself.

    One thing I definitely don't agree with is:

  • WorkOS

    The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

  • mkdkr

    mkdkr = Makefile + Docker

    I just put it in a small solution that I can run in a lot of CI platforms.

    https://github.com/rosineygp/mkdkr

    in another words: https://xkcd.com/927/

  • oil

    Oils is our upgrade path from bash to a better language and runtime. It's also for Python and JavaScript users who avoid shell!

    I think what you are getting at is a "staged execution model", and I agree.

    GNU make actually has this, but it's done poorly. It has build STEPS in the shell language, but the build GRAPH can be done in the Make language, or even Guile scheme.

    ----

    I hope to add the "missing declarative part" to shell with https://www.oilshell.org.

    So the build GRAPH should be described as you say. It's declarative, but you need metaprogramming, which is procedural, akin to generating a Ninja file.

    And then the build STEPS are literally shell. Shell is a lot better than Python for this use case! e.g. for invoking cmopilers and other tools.

    I hinted at this a bit in a previous thread: https://news.ycombinator.com/item?id=25343716

    And this current comment https://lobste.rs/s/k0qhfw/modern_ci_is_too_complex_misdirec...

    Comments welcome!

  • runner

    The Runner for GitHub Actions :rocket:

  • actions-runner-controller

    Kubernetes controller for GitHub Actions self-hosted runners

NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts