pipeline VS artifact-manager-s3-plugin

Tekton

Tekton is an open-source framework for creating efficient CI/CD systems. This empowers developers to seamlessly construct, test, and deploy applications across various cloud environments and on-premise setups.

Despite other alternatives like Circle CI, Travis CI, GitLab CI or even self-hosted options using open-source projects like Tekton or Argo Workflow, the reason for choosing GitHub Actions was straightforward: GitHub Actions, in conjunction with the GitHub ecosystem, offers a user-friendly experience and access to a rich software marketplace.

[2]: https://github.com/tektoncd/pipeline/issues/5507#issuecommen...

Drone, or Tekton, Argo Workflows if you’re on k8s

If you want a step up, I would recommend trying out Tekton Pipelines. It’s a very popular ci tool, and it runs on Kubernetes. Yes, this would involve setting up a Kubernetes cluster but please don’t run for the hills! You can setup a Kubernetes cluster and install Tekton on top of it with minimal setup using minikube (see here. This would be a great joint exercise as it will give you a bit of Kubernetes understanding alongside it, and the mechanisms of Tekton are a little trickier than GitHub actions imo. It’s all much the same though.

I've recently found this quote by Kelsey Hightower:

"I'm convinced the majority of people managing infrastructure just want a PaaS. The only requirement: it has to be built by them."

Source: https://twitter.com/kelseyhightower/status/85193508753294540...

In the last few weeks, I've experimented a bit with Flux (https://fluxcd.io/), Tekton (https://tekton.dev/) and Cloud Native Buildpacks (https://buildpacks.io/) on how to provide K8s powered git push deployments without using a dedicated CI/CD server.

My project is still in early alpha stage and just a proof of concept :-) My vision is to expand it into an Open Source PaaS in the future.

Do you think the above quote is true? What does an open source PaaS need to be like in order to be accepted by software developers?

Some other projects have been discontinued in the past (like Flynn or Deis) or were created before the Kubernetes era.

Is it the right direction to provide a Heroku like solution based on K8s or is it better to provide an Open Source Infrastructure as Code library with building blocks to avoid everything from scratch?

I see. The only option I'm aware of is https://github.com/jenkinsci/artifact-manager-s3-plugin (S3 API compatible).

So, yep done all this and completely agree. I set this all up with Terraform, Docker, and JCasC. However with our 80 person org and around 30 active repos (all with multibranch pipelines) ,200-300 builds a day; I'm seeing reboot times in the 15-20 minutes range (which hurts.) Also, if you don't drain the Jenkins instance prior to replacement then the currently running jobs fail; causing more issues (need to automate this anyone got a script?). Other complaints, plugins break ALL THE TIME so we really have to do some QA before sending a new Jenkins container out to prod. The S3 Artifact plugin doesn't do multipart uploads which limits the maximum size and performance on stash/archiveArtifact (patch here https://github.com/jenkinsci/artifact-manager-s3-plugin/pull/141; could still use a patch to stream the zip/tarball creation). The downstream build feature is really powerful but the blue ocean ui is iffy on it (sometimes it shows downstream builds; sometimes not). The ec2 plugin works but I've never gotten the spot-fleet plugin to work properly (specifically interested in being able to choose alternately sized instances based on spot price and availability). If you use spot-instances with the ec2 plugin; there's no clear way to retry the build (would love a tutorial or script that accomplishes the steps needed on this. i.e. trapping the signal coming from the OS and turning it into an exception in the pipeline). Overall, it's nice once you get here; but it's a metric ton of work... and there's still a bunch of problems. Really wish there was something easier to maintain and grow with a cloud first approach (i.e. rolling upgrades, object store support, fleet support).