Our great sponsors
-
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
-
-
-
WorkOS
The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
- the caching works across branches, so for example merging a feature branch to master, if nothing changes the build on master will be very quick
I created something similar to nix-cache for gitlab, but I had to create a dedicated runner running NixOS.
If I could use NixOS for deployment, at that point I would just point the same binary cache to the machine and use the same derivation to build the app. Because the app was already build by CI, it would just download the compiled version. No need for artifactory or similar. In that scenario (you using poetry) you probably would just use poetry2nix to generate the application.
If the OS is not NixOS, but you still want to deploy via nix, then IMO this[2] looks interesting, basically it packages everything in self extracting archive. That you can extract and then run the app.
Other alternatives are these bundlers[3], which includes building toArx (works in a way similar to the previous one but pretends everything is in a single file), RPM, DEB, docker (you would have more control over it if you would use the code directly instead of a bundler though)
And the last option (probably the most obvious one) is that you can simply just use the tool to build the package. Since you're using poetry, then you can generate a wheel from it.
[1] https://github.com/takeda/nix-cde/blob/master/contrib/gitlab...
[2] https://github.com/Ninlives/relocatable.nix
[3] https://github.com/NixOS/bundlers/blob/master/flake.nix
> Does it make sense to use nix during the image build process to precisely define dependencies I want to have in the resulting container?
Using a dockerfile with a step like "RUN nix ..."? You can, but it strikes me as a cumbersome way of doing things. Mitchell H describes this way of doing things here: https://mitchellh.com/writing/nix-with-dockerfiles
Whereas, I'd reckon the more idiomatic thing to do is to build the Docker image with Nix code. -- You're going to get a precisely defined environment to share across workstation/CI/etc. Some example Nix code for building Docker images is here https://github.com/NixOS/nixpkgs/blob/master/pkgs/build-supp...
One project cut their CI time from 18m to 3m: https://github.com/awkward-squad/hasql-interpolate/actions. I wonder who will see the biggest cut!
Note that when PRs merge to the default branch, their cache doesn't come with them. This is how GitHub Action's cache works, as a security measure. However: subsequent rebuilds will, and PRs off the default branch will too.
You may find something like node2nix helpful (https://github.com/svanderburg/node2nix). This converts your package.json into a Nix expression that can then be cached. You're right that it does require some setup and a bit of Nix knowledge but could yield significant benefits and take a good chunk out of that 20 minutes.
Another option might be to use pnpm instead of Yarn and cache your pnpm dependencies. pnpm actually works a bit like Nix in that it creates a pnpm-lock.yaml file with content-based hashes for the full package.json dependency tree. This enables it to quickly determine which parts of the dependency tree it needs to build and which are already available.
- the caching works across branches, so for example merging a feature branch to master, if nothing changes the build on master will be very quick
I created something similar to nix-cache for gitlab, but I had to create a dedicated runner running NixOS.
If I could use NixOS for deployment, at that point I would just point the same binary cache to the machine and use the same derivation to build the app. Because the app was already build by CI, it would just download the compiled version. No need for artifactory or similar. In that scenario (you using poetry) you probably would just use poetry2nix to generate the application.
If the OS is not NixOS, but you still want to deploy via nix, then IMO this[2] looks interesting, basically it packages everything in self extracting archive. That you can extract and then run the app.
Other alternatives are these bundlers[3], which includes building toArx (works in a way similar to the previous one but pretends everything is in a single file), RPM, DEB, docker (you would have more control over it if you would use the code directly instead of a bundler though)
And the last option (probably the most obvious one) is that you can simply just use the tool to build the package. Since you're using poetry, then you can generate a wheel from it.
[1] https://github.com/takeda/nix-cde/blob/master/contrib/gitlab...
[2] https://github.com/Ninlives/relocatable.nix
[3] https://github.com/NixOS/bundlers/blob/master/flake.nix
- the caching works across branches, so for example merging a feature branch to master, if nothing changes the build on master will be very quick
I created something similar to nix-cache for gitlab, but I had to create a dedicated runner running NixOS.
If I could use NixOS for deployment, at that point I would just point the same binary cache to the machine and use the same derivation to build the app. Because the app was already build by CI, it would just download the compiled version. No need for artifactory or similar. In that scenario (you using poetry) you probably would just use poetry2nix to generate the application.
If the OS is not NixOS, but you still want to deploy via nix, then IMO this[2] looks interesting, basically it packages everything in self extracting archive. That you can extract and then run the app.
Other alternatives are these bundlers[3], which includes building toArx (works in a way similar to the previous one but pretends everything is in a single file), RPM, DEB, docker (you would have more control over it if you would use the code directly instead of a bundler though)
And the last option (probably the most obvious one) is that you can simply just use the tool to build the package. Since you're using poetry, then you can generate a wheel from it.
[1] https://github.com/takeda/nix-cde/blob/master/contrib/gitlab...
[2] https://github.com/Ninlives/relocatable.nix
[3] https://github.com/NixOS/bundlers/blob/master/flake.nix
I think it's mostly just a prototype: https://github.com/elohmeier/flakehub