Our great sponsors
-
-
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
-
-
-
WorkOS
The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
-
-
https://github.com/woodpecker-ci/woodpecker
> We are uding docker-in-docker at the moment
You can also run a "less privileged" container with all the features of Docker by using rootless buildkit in Kubernetes. Here are some examples:
https://github.com/moby/buildkit/tree/master/examples/kubern...
https://github.com/moby/buildkit/blob/master/examples/kubern...
It's also possible to run dedicated buildkitd workers and connect to them remotely.
> On a previous team I had used Concourse CI to some extent, but I wasn’t really blown away by the experience. Travis and Circle were mentioned. I was a fool. I should have committed to seriously researching some of the contenders and making a more informed decision, but I lacked the willpower and the discernment.
The whole post can be summed up as he had very little CICD experience. Made lots of beginner mistakes, which is easy to do in Jenkins. Then decided to write a post where all his complaints about Jenkins are not only wrong but are the issues that plague all the other CICD tools.
> So instead of writing Bash directly, you’re writing Bash inside Groovy
Why are you doing that? You have a fully featured programming language and you are running `sh('npm install')`. You could do this instead https://github.com/DontShaveTheYak/jenkins-std-lib/blob/mast... . How is bash inside of YAML better?
> The trouble is: Groovy is a much, much worse language for executing commands than Bash. Bash is interpreted, has a REPL that is great for experimentation, does require a ton of imports, and has lightweight syntax. Groovy has none of these things.
Groovy has a language server, linters and a vscode IDE plugins. They are probably not as stable or full featured as the bash ones, but they are available and very few take advantage of them. Again, how is YAML+Bash better?
> The way that developers test their Groovy steps is by triggering a job on the remote Jenkins server to run them. The feedback loop is 2 orders of magnitude slower than it is for just executing Bash locally.
This is a rookie mistake. For about 60-75% of pipelines you can run them locally in a docker container on your local machine. You can even set up hot code reload so as you change your pipeline the Jenkins reloads it. You can also configure the job to kick off a build when it reloads the code. When Jenkins is configured correctly it has the fastest feedback loop of any CICD tool on the market. GitHub actions comes in a close second since it can also be run locally but you cant run a "clone" of what you run in production, like having the same secrets, so it gets second place. Beside Jenkins and GitHub actions, I dont know of any solutions for the other tools.
You can run a GitHub action on Jenkins. It's a very deep and complex system. It's like an iceberg and so many engineers dont leave the surface before deciding it sucks and one of the YAML CICD tools is better. Sure the YAML alternatives are EASY to get started with and to do basic stuff with. But they are Terrible at anything complex. While Jenkins is not easy to get started with, once mastered, you can build complex pipelines with ease.
I get that I'm a Jenkins fanboy. Most of the things I mentioned above, I either contribute to or I'm the author of. I know Jenkins has issues. I know it has hurt lots of people, I read the complaints online. But it's still the best out there. The best software in the world is not written in bash or yaml and the same is true of the best CICD pipelines in the world. It's a shame very few people get to see/use those pipelines.
It’s possible to do the ‘uses’ trick but there’s a few extras needed admittedly (branch name and path to file)
include: https://gitlab.com/awesome-project/raw/main/.before-script-template.yml
At a recent job, we had slightly different containers for local dev; our backend containers (for a Go app) had Air [1] installed for live reloading, plus Delve [2] running inside the container for VS Code's debugger to connect to. We also had a frontend container for local dev, which didn't get deployed as a container, just as static files.
[1] https://github.com/cosmtrek/air
[2] https://github.com/go-delve/delve/
At a recent job, we had slightly different containers for local dev; our backend containers (for a Go app) had Air [1] installed for live reloading, plus Delve [2] running inside the container for VS Code's debugger to connect to. We also had a frontend container for local dev, which didn't get deployed as a container, just as static files.
[1] https://github.com/cosmtrek/air
[2] https://github.com/go-delve/delve/
cloud is so good now it’s hard to justify not doing something bespoke. ec2 spot is insanely cheaper than turnkey cicd, and better in almost every way.
i’m delighted to pay 30% over infra cost for convenience, but not 500%. and it better actually be convenient, not just have a good landing page and sales team.
this month i learned localzones have even better spot prices. losangeles-1 is half the spot price of us-west-2.
for a runner, do something like this, but react to an http call instead of a s3 put[1].
for a web ui do something like this[2].
s3, lambda, and ec2 spot are a perfect fit for cicd and a lot more.
1. https://github.com/nathants/libaws/tree/91b1c27fc947e067ed46...
2. https://github.com/nathants/aws-exec/tree/e68769126b5aae0e35...
cloud is so good now it’s hard to justify not doing something bespoke. ec2 spot is insanely cheaper than turnkey cicd, and better in almost every way.
i’m delighted to pay 30% over infra cost for convenience, but not 500%. and it better actually be convenient, not just have a good landing page and sales team.
this month i learned localzones have even better spot prices. losangeles-1 is half the spot price of us-west-2.
for a runner, do something like this, but react to an http call instead of a s3 put[1].
for a web ui do something like this[2].
s3, lambda, and ec2 spot are a perfect fit for cicd and a lot more.
1. https://github.com/nathants/libaws/tree/91b1c27fc947e067ed46...
2. https://github.com/nathants/aws-exec/tree/e68769126b5aae0e35...
