Incremental Builds for Haskell with Bazel

This page summarizes the projects mentioned and recommended in the original post on news.ycombinator.com

Our great sponsors
  • InfluxDB - Power Real-Time Data Analytics at Scale
  • WorkOS - The modern identity platform for B2B SaaS
  • SaaSHub - Software Alternatives and Reviews
  • bazel_rules_qt

    Bazel rules for Qt5

  • I migrated a mid-size polyglot project from Makefiles to Bazel and C++ was a large component of the project.

    Some obstacles:

    1. Building with QT5 MOC & UI files. There is a great library[0] for it but it has hardcoded paths to the QT binaries and header files assuming a system-wide installation. I had to patch the rule to point to our QT location. Then it worked fine.

    2. There is no rule to build a fully static library[1]. Since we were shipping a static library in our Makefile system, that was somewhat annoying.

    3. We were using system links like `$PROJECT_ROOT/links/GCC/vX.Y.Z/ -> /opt/gcc/...` to point to all the build tools, but these didn't work in Bazel I think because it required absolute paths for any binaries it calls. We ended up putting them in a .bazelrc but we would need a different one for Windows and Linux.

    4. Not good integration with IDEs

    Ultimately we did not keep using Bazel because we were building Python binaries and py_binary was too slow on Windows. And we didn't have enough time to write a PyInstaller rule.

    [0]: https://github.com/justbuchanan/bazel_rules_qt

    [1]: https://github.com/bazelbuild/bazel/issues/1920

  • Bazel

    a fast, scalable, multi-language and extensible build system

  • I migrated a mid-size polyglot project from Makefiles to Bazel and C++ was a large component of the project.

    Some obstacles:

    1. Building with QT5 MOC & UI files. There is a great library[0] for it but it has hardcoded paths to the QT binaries and header files assuming a system-wide installation. I had to patch the rule to point to our QT location. Then it worked fine.

    2. There is no rule to build a fully static library[1]. Since we were shipping a static library in our Makefile system, that was somewhat annoying.

    3. We were using system links like `$PROJECT_ROOT/links/GCC/vX.Y.Z/ -> /opt/gcc/...` to point to all the build tools, but these didn't work in Bazel I think because it required absolute paths for any binaries it calls. We ended up putting them in a .bazelrc but we would need a different one for Windows and Linux.

    4. Not good integration with IDEs

    Ultimately we did not keep using Bazel because we were building Python binaries and py_binary was too slow on Windows. And we didn't have enough time to write a PyInstaller rule.

    [0]: https://github.com/justbuchanan/bazel_rules_qt

    [1]: https://github.com/bazelbuild/bazel/issues/1920

  • InfluxDB

    Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

    InfluxDB logo
  • toolchains_llvm

    LLVM toolchain for bazel

  • Yeah the cross-compilation thing is definitely a rough spot. I have one project that's able to work around it via extensive hacks with macros, but at some point I'll need to do it "the right way."

    Honestly if the docs had a canonical example of e.g. using unix_cc_toolchain_config (example: [0]) + Bootlin to compile for aarch64, it'd probably go a long way to making things understandable. Because say what you will about the old CROSSTOOL approach, at least there was a nice tutorial for it.

    [0] https://github.com/grailbio/bazel-toolchain/blob/f14a8a5de8f...

  • gcc-toolchain

    A fully-hermetic Bazel GCC toolchain for Linux.

  • rules_python

    Bazel Python Rules

  • Python support in Bazel now looks more promising with `rules_python`: https://github.com/bazelbuild/rules_python

    `rules_go` to my understanding is great too.

    Over years, Bazel is not as opinionated as before, mostly because adoptions in different orgs force it to be so.

  • llvm-project

    The LLVM Project is a collection of modular and reusable compiler and toolchain technologies.

  • The canonical build system for LLVM is CMake, but you can also build LLVM with Bazel now. That might give you a good idea of how to do it. The Bazel support code is in https://github.com/llvm/llvm-project/tree/main/utils/bazel

  • nixpkgs

    Nix Packages collection & NixOS

  • I've mostly associated Tweag with Nix-related content, so this is an interesting change.

    That said, it's interesting to me that there hasn't been a better Nix-Bazel bridge. Right now the story for packaging Bazel projects in Nix is really awful. You basically have Bazel run its "deps" phase, and all that stuff gets stored as a singular, gargantuan, fixed output derivation, and if you ever try to change the deps, you have to just know that you have to twiddle this magic hash [1] or it'll happily go on with the cached deps. But then, of course, every build is from scratch.

    A "real" Bazel story for Nix would integrate the two binary caches together, so that Nix would be aware of what Bazel was building, and the individual cached elements of it could go into the Nix store as separate entries— then you'd be able to actually get incremental builds. But I assume this would require Bazel's binary cache implementation to be pluggable.

    [1]: https://github.com/NixOS/nixpkgs/blob/21321a6381fd8d3660fe1c...

  • WorkOS

    The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

    WorkOS logo
NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts