Our great sponsors
-
-
-
WorkOS
The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
-
-
-
-
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
The cache action section focuses on GHES which is a shame, since the GitHub.com cache action is also pretty bad.
- can't manually delete cache (although they're considering changing that[0])
- only saves cache at the end of a workflow if it succeeds (would be solved with CircleCI's approach of having a save-cache step and a restore-cache step)
- Cache is super slow for self-hosted runners, so it makes more sense to have a local cache instead of using the action
- only 5gb of storage size. This was supposed to be increased via billable cache storage[1], but it's been on the backburner since July 2020
- In addition to the above, you can't use a different storage backend on the official action (which would allow storing over 5GB of cache via your own storage). The best workaround is to use a user-provided action which utilizes the s3 api[2].
0: https://github.com/actions/cache/issues/632
1: https://github.com/github/roadmap/issues/66
2: https://github.com/actions/cache/issues/354#issuecomment-854...
2>
The cache action section focuses on GHES which is a shame, since the GitHub.com cache action is also pretty bad.
- can't manually delete cache (although they're considering changing that[0])
- only saves cache at the end of a workflow if it succeeds (would be solved with CircleCI's approach of having a save-cache step and a restore-cache step)
- Cache is super slow for self-hosted runners, so it makes more sense to have a local cache instead of using the action
- only 5gb of storage size. This was supposed to be increased via billable cache storage[1], but it's been on the backburner since July 2020
- In addition to the above, you can't use a different storage backend on the official action (which would allow storing over 5GB of cache via your own storage). The best workaround is to use a user-provided action which utilizes the s3 api[2].
0: https://github.com/actions/cache/issues/632
1: https://github.com/github/roadmap/issues/66
2: https://github.com/actions/cache/issues/354#issuecomment-854...
2>
If you use Jenkins and want to try actions then check out https://github.com/DontShaveTheYak/jenkins-std-lib
It let's you run actions on top of Jenkins.
Yes, exactly. All of our build pipelines for a repository are included in the .github folder in the root of the repo. It makes it easier for team members to feel comfortable making changes and submitting a PR for them. You can setup an ACT container so you can test GitHub Action changes locally before pushing them too (see https://github.com/nektos/act )
> Out of curiosity, how reliable do you find the environment cleanup is?
Turnstyle?
We've had similar issues with Terraform deployments.
https://github.com/softprops/turnstyle
Related to 3.3 (You can’t restart a single job of a workflow), is there any reason why the numeric id of a specific job run isn't available anywhere? There's GITHUB_RUN_ID, but that only identifies the workflow run, not the individual jobs, and this isn't unique across run/job restarts. Services like https://coveralls.io/ need an actually unique job run id, and they could also use the numeric id to link to the specific job run in a build matrix.
To clarify this further, in https://github.com/xmonad/xmonad/actions/runs/1201348600, GITHUB_RUN_ID=1201348600, GITHUB_RUN_NUMBER=168, but there's no way to get the 3514203530 out of https://github.com/xmonad/xmonad/runs/3514203530.
We use https://github.com/actions-runner-controller/actions-runner-... to auto scale on EKS. It also allows you to use a GitHub App for the runner registration instead of personal access tokens. Also it seems like the project is receives support from GitHub because they’re getting early access to test out features.
We use https://github.com/actions-runner-controller/actions-runner-... to auto scale on EKS. It also allows you to use a GitHub App for the runner registration instead of personal access tokens. Also it seems like the project is receives support from GitHub because they’re getting early access to test out features.
To be honest I've always found the best approach with most of these systems is to checking your magic as shell-scripts inside your repository.
Then you're much much more portable. Want to run tests? Run ".ci/tests.sh", want to generate artifacts "make", or ".ci/build.sh".
All systems, be they github actions, jenkins, gitlab-runners, and everything else allow you to clone/update your repository and run something from within it. Which keeps things mostly portable.
I put together a simple github action a long time ago, but now of course I realize it is overkill:
https://github.com/skx/github-action-tester/
3.5 I was excited to see action composition until I realized that the composed action will still be showed as one step and that it can just compose actions instead of whole workflows.
I want to share whole jobs across my repos, including matrixes and all. I could wrap some common workflows into one action, but then the whole complexity will be hidden in a single step.
So I resorted to creating GHA Templates and manually synching them with a CLI tool: https://github.com/fregante/ghat
Sorry, it was with `only:changes` and `needs`. Take a look at this issue[0] and this pipeline[1]. I've failed to find the failure mode in the documentation, so I suppose it may have been fixed since then - but we've developed an in-house workaround in the meantime that I'd trust a lot more than anything coming out from Gitlab.
--------
> A monorepo isn't "all but the most simple use-cases", it's usually a fairly complex usecase, and Gitlab have a myriad of ways to make monorepo CI easier - dynamic pipelines, remote triggers, includes, etc.
And every single feature you've mentioned here has a bug when combined with something else. That's the problem with Gitlab CI: everything works in isolation, but nothing composes properly.
Take includes: they don't work with anchors, so you couldn't have a generic template rules in the "main" file getting reused in the included files. This makes sense though! Anchors are a yaml feature. So gitlab added their own pseudo-anchors, called `extends`. You'd assume a smart, context-aware merge to happen, but no! Gitlab decided to go with a dumb object merge. Because the `script` step is a list of string, if both the parent and the child specify a `script`, only the child's will be used! Gitlab has a `before_script` step which can be used to workaround the issue for single levels of inheritance, but anything more complex ends up in a dead end. This feels like a feature that's been bolted on without any sort of design work.
[0]: https://gitlab.com/gitlab-org/gitlab/-/issues/31310
[1]: https://gitlab.com/ensc/test-ci-needs/-/pipelines/78713602
Sorry, it was with `only:changes` and `needs`. Take a look at this issue[0] and this pipeline[1]. I've failed to find the failure mode in the documentation, so I suppose it may have been fixed since then - but we've developed an in-house workaround in the meantime that I'd trust a lot more than anything coming out from Gitlab.
--------
> A monorepo isn't "all but the most simple use-cases", it's usually a fairly complex usecase, and Gitlab have a myriad of ways to make monorepo CI easier - dynamic pipelines, remote triggers, includes, etc.
And every single feature you've mentioned here has a bug when combined with something else. That's the problem with Gitlab CI: everything works in isolation, but nothing composes properly.
Take includes: they don't work with anchors, so you couldn't have a generic template rules in the "main" file getting reused in the included files. This makes sense though! Anchors are a yaml feature. So gitlab added their own pseudo-anchors, called `extends`. You'd assume a smart, context-aware merge to happen, but no! Gitlab decided to go with a dumb object merge. Because the `script` step is a list of string, if both the parent and the child specify a `script`, only the child's will be used! Gitlab has a `before_script` step which can be used to workaround the issue for single levels of inheritance, but anything more complex ends up in a dead end. This feels like a feature that's been bolted on without any sort of design work.
[0]: https://gitlab.com/gitlab-org/gitlab/-/issues/31310
[1]: https://gitlab.com/ensc/test-ci-needs/-/pipelines/78713602