opsindev.news VS gitlab-runner

The search works great. I'm using MkDocs with Material as my personal handbook because of the simplicity -- for example, I usually remember great articles in conversations but always forget their location. Since I started writing my newsletter https://opsindev.news/ including an MkDocs web archive, I can share interesting URLs way faster :) Or let folks discover it by themselves, using the search.

Configuration in https://gitlab.com/dnsmichi/opsindev.news/-/blob/main/mkdocs...

Material for MkDocs also has an insiders build, accessible through sponsorship. https://squidfunk.github.io/mkdocs-material/insiders/ These features add more value to MkDocs -- I initially joined to get GDPR-compliant cookie banners and stayed to support a great project.

Thanks for flagging. opsindev.news is hosted on GitLab Pages using Let's Encrypt. Maybe the TLS ciphers or versions do not match. Which error message do you see, browser/OS and location may help (if you want to share)

If the MkDocs website does not work, suggest the following workarounds: Newsletter issues in [0] as markdown files, example [1] or the newsletter archive on Buttondown [2].

I'm using Buttondown to send the newsletter, MkDocs serves as web-searchable archive. Kudos to Michael Hausenblas here for the idea, he publishes the o11y.news newsletter.

[0] https://gitlab.com/dnsmichi/opsindev.news/-/tree/main/docs/a...

[1] https://gitlab.com/dnsmichi/opsindev.news/-/blob/main/docs/a...

[2] https://buttondown.email/opsindev.news/

#!/bin/bash # ### Script to initialize a GitLab runner on an existing AWS EC2 instance with NVME disk(s) # # - script is not interactive (can be run as user_data) # - will reboot at the end to perform NVME mounting # - first NVME disk will be used for GitLab custom cache # - last NVME disk will be used for Docker data (if only one NVME, the same will be used without problem) # - robust: on each reboot and stop/start, disks are mounted again (but data may be lost if stop and then start after a few minutes) # - runner is tagged with multiple instance data (public dns, IP, instance type...) # - works with a single spot instance # - should work even with multiple ones in a fleet, with same user_data (not tested for now) # # /!\ There is no prerequisite, except these needed variables : MAINTAINER=zenika RUNNER_NAME="majestic-runner" GITLAB_URL=https://gitlab.com/ GITLAB_TOKEN=XXXX # prepare docker (re)install sudo apt-get -y install apt-transport-https ca-certificates curl gnupg lsb-release sysstat curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list >/dev/null sudo apt-get update # needed to use the docker.list # install gitlab runner curl -L "https://packages.gitlab.com/install/repositories/runner/gitlab-runner/script.deb.sh" | sudo bash sudo apt-get -y install gitlab-runner # create NVME initializer script cat </home/ubuntu/nvme-initializer.sh #!/bin/bash # # To be run on each fresh start, since NVME disks are ephemeral # so first start, start after stop, but not on reboot # inspired by https://stackoverflow.com/questions/45167717/mounting-a-nvme-disk-on-aws-ec2 # date | tee -a /home/ubuntu/nvme-initializer.log ### Handle NVME disks # get NVME disks bigger than 100Go (some small size disk may be there for root, depending on server type) NVME_DISK_LIST=\$(lsblk -b --output=NAME,SIZE | grep "^nvme" | awk '{if(\$2>100000000000)print\$1}' | sort) echo "NVME disks are: \$NVME_DISK_LIST" | tee -a /home/ubuntu/nvme-initializer.log # there may be 1 or 2 NVME disks, then we split (or not) the mounts between GitLab custom cache and Docker data export NVME_GITLAB=\$(echo "\$NVME_DISK_LIST" | head -n 1) export NVME_DOCKER=\$(echo "\$NVME_DISK_LIST" | tail -n 1) echo "NVME_GITLAB=\$NVME_GITLAB and NVME_DOCKER=\$NVME_DOCKER" | tee -a /home/ubuntu/nvme-initializer.log # format disks if not sudo mkfs -t xfs /dev/\$NVME_GITLAB | tee -a /home/ubuntu/nvme-initializer.log || echo "\$NVME_GITLAB already formatted" # this may already be done sudo mkfs -t xfs /dev/\$NVME_DOCKER | tee -a /home/ubuntu/nvme-initializer.log || echo "\$NVME_DOCKER already formatted" # disk may be the same, then already formated by previous command # mount on /gitlab-host/ and /var/lib/docker/ sudo mkdir -p /gitlab sudo mount /dev/\$NVME_GITLAB /gitlab | tee -a /home/ubuntu/nvme-initializer.log sudo mkdir -p /gitlab/custom-cache sudo mkdir -p /var/lib/docker sudo mount /dev/\$NVME_DOCKER /var/lib/docker | tee -a /home/ubuntu/nvme-initializer.log ### reinstall Docker (which data may have been wiped out) # docker (re)install sudo apt-get -y reinstall docker-ce docker-ce-cli containerd.io docker-compose-plugin | tee -a /home/ubuntu/nvme-initializer.log echo "NVME initialization succesful" | tee -a /home/ubuntu/nvme-initializer.log EOF # set NVME initializer script as startup script sudo tee /etc/systemd/system/nvme-initializer.service >/dev/null <

GitLab team member here, thanks for sharing.

> Still not a big fan of how stiff Yaml pipelines feel in Gitlab CI

Maybe the pipeline editor in "Build > Pipeline editor" can help with live linting, or more advanced features such as parent-child pipelines or merge trains.

If you need tips for optimizing the CI/CD pipeline, suggest following these tips in the docs https://docs.gitlab.com/ee/ci/pipelines/pipeline_efficiency.... or a few more tips in my recent talk "Efficient DevSecOps pipelines in cloud-native world", slides from Chemnitz Linux Days 2023 in https://docs.google.com/presentation/d/1_kyGo_cWi5dKyxi3BfYj...

> and that tickets for what seems like a simple feature [1] hang around for years, but it is nice.

Thanks for sharing. (FYI for everyone) The linked issue suggests a Docker cache cleanup script, which might be helpful. https://gitlab.com/gitlab-org/gitlab-runner/-/issues/27332#n... -> https://docs.gitlab.com/runner/executors/docker.html#clear-t...

If only competitors could do better...

https://gitlab.com/gitlab-org/gitlab-runner/-/issues/2797

GitLab MR for the CACHE_COMPRESSION_LEVEL implementation

And in general just making caching stuff easier. I feel like it is unnecessarily complicated for example to cache apt-get in Gitlab which I assume makes most people not do it.

https://gitlab.com/gitlab-org/gitlab-runner/-/issues/991#not...