v4 VS gitlab-runner

#!/bin/bash # list-tags.sh # List all Gitlab tags for a given project. # $CI_PROJECT_ID is a Gitlab pre-defined variable # $GITLAB_TOKEN is a project access token that needs to be # a defined CI/CD variable. GITLAB_API_URL="https://gitlab.com/api/v4/projects/${CI_PROJECT_ID}" curl -k --header "PRIVATE-TOKEN: $GITLAB_TOKEN" "${GITLAB_API_URL}/repository/tags"

# ~/.npmrc @doppelmutzi:registry=https://gitlab.com/api/v4/projects//packages/npm/ //gitlab.com/api/v4/projects//packages/npm/:_authToken=

GITLAB_API_URL = "https://gitlab.com/api/v4" PRIVATE_TOKEN = "" NAMESPACE = "mycompany" PROJECT_NAME = "test-project"

import requests app = FastAPI() # GitLab API configuration GITLAB_API_URL = "https://gitlab.com/api/v4" PRIVATE_TOKEN = "" PROJECT_ID = "vinod827_1391/awesome" def grant_developer_access(username: str): # Get user details from GitLab user_url = f"{GITLAB_API_URL}/users?username={username}" response = requests.get(user_url, headers={"PRIVATE-TOKEN": PRIVATE_TOKEN}) user_data = response.json() if not user_data: raise HTTPException(status_code=404, detail=f"User {username} not found.") user_id = user_data[0]["id"] # Grant developer access to the project access_data = {"user_id": user_id, "access_level": 30} # 30 is the access level for developer access_url = f"{GITLAB_API_URL}/projects/{PROJECT_ID}/access_requests" response = requests.post(access_url, headers={"PRIVATE-TOKEN": PRIVATE_TOKEN}, json=access_data) print('response->', response) if response.status_code != 201: raise HTTPException(status_code=response.status_code, detail=response.text)

Invoke-RestMethod -Uri "https://gitlab.com/api/v4/projects/12345/repository/tags" -Headers $headers -Method POST -Body $jsonBody

Hello everyone i'm trying to install Librewolf on my M1 MBA running fedora39 with gnome on wayland, I downloaded librewolf's compressed archive from https://gitlab.com/api/v4/projects/44042130/packages/generic/librewolf/120.0.1-1/librewolf-120.0.1-1-linux-arm64-package.tar.bz2

repositories { maven { url "https://gitlab.com/api/v4/projects/PROJEC-ID/packages/maven" credentials(HttpHeaderCredentials) { name = "Private-Token" value = gitLabPrivateToken } authentication { header(HttpHeaderAuthentication) } } } }

Pagina da Gitlab: https://gitlab.com/api/v4/projects/44042130/packages/generic/librewolf/116.0-1/librewolf-116.0-1-windows-x86_64-setup.exe

.base: image: alpine:3.18 before_script: - echo "Project is $CI_PROJECT_NAME" script: - apk update - apk --no-cache add git openssl ca-certificates curl perl https://$GITLAB_USER_NAME:$SVC_ACCESS_TOKEN@$CI_SERVER_HOST/$CI_PROJECT_PATH.git - mkdir -p /usr/local/share/ca-certificates/extra - openssl s_client -connect ${CI_SERVER_HOST}:${CI_SERVER_PORT} -servername ${CI_SERVER_HOST} -showcerts /dev/null | sed -e '/-----BEGIN/,/-----END/!d' | tee "/usr/local/share/ca-certificates/${CI_SERVER_HOST}.crt" >/dev/null - update-ca-certificates - curl --location --output /usr/local/bin/release-cli "https://gitlab.com/api/v4/projects/gitlab-org%2Frelease-cli/packages/generic/release-cli/latest/release-cli-linux-amd64" - chmod +x /usr/local/bin/release-cli - release-cli -v Versioning: stage: Version extends: - .base script: - if [[ "$CI_COMMIT_BRANCH" == *"feature/"* ]]; then IMAGE_TAG="nightly" ; elif [ "$CI_COMMIT_BRANCH" == "develop" ]; then IMAGE_TAG="devel"; else echo "Master/Main branch"; fi - echo "Version found is ${IMAGE_TAG}" allow_failure: false rules: - if: '$CI_COMMIT_BRANCH =~ /^(develop|main|feature.*$)$/' artifacts: paths: - $IMAGE_TAG expire_in: 1 minute

image: name: hashicorp/terraform:1.5 entrypoint: - '/usr/bin/env' - 'PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin' stages: - Test and Lint - Build and Push - Staging Plan - Staging Apply - Staging Update App - Production Plan - Production Apply - Destroy before_script: - alias convert_report="jq -r '([.resource_changes[]?.change.actions?]|flatten)|{\"create\":(map(select(.==\"create\"))|length),\"update\":(map(select(.==\"update\"))|length),\"delete\":(map(select(.==\"delete\"))|length)}'" - apk add openssh-client - eval $(ssh-agent -s) - echo "$SSH_PRIVATE_KEY" | tr -d '\r' | ssh-add - - mkdir -p ~/.ssh - chmod 700 ~/.ssh - echo "$SSH_PRIVATE_KEY" > ~/.ssh/id_rsa - ls ~/.ssh Test and Lint: image: docker:19.03.5 .............. Validate Terraform: stage: Test and Lint script: - echo "Validate Terraform" - cd deploy/ - terraform init --backend=false - terraform validate rules: - if: '$CI_MERGE_REQUEST_TARGET_BRANCH_NAME =~ /^(main|production)$/ || $CI_COMMIT_BRANCH =~ /^(main|production)$/' Build and Push: stage: Build and Push ............... Staging Plan: stage: Staging Plan script: - echo "Run Terraform Plan for Staging" - cd deploy/ - export TF_VAR_image_api=$CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA - export GITLAB_ACCESS_TOKEN=XXXXXXXXXXXXXXXXXXXXXXXXX - export STATE_NAME=Staging - terraform init -backend-config="address=https://gitlab.com/api/v4/projects/47021553/terraform/state/$STATE_NAME" -backend-config="lock_address=https://gitlab.com/api/v4/projects/47021553/terraform/state/$STATE_NAME/lock" -backend-config="unlock_address=https://gitlab.com/api/v4/projects/47021553/terraform/state/$STATE_NAME/lock" -backend-config="username=XXXXXXXXX" -backend-config="password=$GITLAB_ACCESS_TOKEN" -backend-config="lock_method=POST" -backend-config="unlock_method=DELETE" -backend-config="retry_wait_min=5" #- terraform workspace select staging || terraform workspace new staging - terraform plan rules: - if: '$CI_COMMIT_BRANCH =~ /^(main|production)$/' Staging Apply: stage: Staging Apply environment: name: Staging url: XXXXXXXXXXXXXXXXX on_stop: Staging Destroy script: - echo "Run Terraform Apply for Staging" - cd deploy/ - export TF_VAR_image_api=$CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA - export GITLAB_ACCESS_TOKEN=XXXXXXXXXXXXXXXXXXXXXXX - export STATE_NAME=Staging - terraform init -backend-config="address=https://gitlab.com/api/v4/projects/47021553/terraform/state/$STATE_NAME" -backend-config="lock_address=https://gitlab.com/api/v4/projects/47021553/terraform/state/$STATE_NAME/lock" -backend-config="unlock_address=https://gitlab.com/api/v4/projects/47021553/terraform/state/$STATE_NAME/lock" -backend-config="username=XXXXXXXXXXXXX" -backend-config="password=$GITLAB_ACCESS_TOKEN" -backend-config="lock_method=POST" -backend-config="unlock_method=DELETE" -backend-config="retry_wait_min=5" - terraform apply -auto-approve -input=false rules: - if: '$CI_COMMIT_BRANCH =~ /^(main|production)$/' Staging Update App: stage: Staging Update App ............... Production Plan: stage: Production Plan ............. Production Apply: stage: Production Apply .......... Staging Destroy: stage: Destroy ............... Production Destroy: stage: Destroy ...............

#!/bin/bash # ### Script to initialize a GitLab runner on an existing AWS EC2 instance with NVME disk(s) # # - script is not interactive (can be run as user_data) # - will reboot at the end to perform NVME mounting # - first NVME disk will be used for GitLab custom cache # - last NVME disk will be used for Docker data (if only one NVME, the same will be used without problem) # - robust: on each reboot and stop/start, disks are mounted again (but data may be lost if stop and then start after a few minutes) # - runner is tagged with multiple instance data (public dns, IP, instance type...) # - works with a single spot instance # - should work even with multiple ones in a fleet, with same user_data (not tested for now) # # /!\ There is no prerequisite, except these needed variables : MAINTAINER=zenika RUNNER_NAME="majestic-runner" GITLAB_URL=https://gitlab.com/ GITLAB_TOKEN=XXXX # prepare docker (re)install sudo apt-get -y install apt-transport-https ca-certificates curl gnupg lsb-release sysstat curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list >/dev/null sudo apt-get update # needed to use the docker.list # install gitlab runner curl -L "https://packages.gitlab.com/install/repositories/runner/gitlab-runner/script.deb.sh" | sudo bash sudo apt-get -y install gitlab-runner # create NVME initializer script cat </home/ubuntu/nvme-initializer.sh #!/bin/bash # # To be run on each fresh start, since NVME disks are ephemeral # so first start, start after stop, but not on reboot # inspired by https://stackoverflow.com/questions/45167717/mounting-a-nvme-disk-on-aws-ec2 # date | tee -a /home/ubuntu/nvme-initializer.log ### Handle NVME disks # get NVME disks bigger than 100Go (some small size disk may be there for root, depending on server type) NVME_DISK_LIST=\$(lsblk -b --output=NAME,SIZE | grep "^nvme" | awk '{if(\$2>100000000000)print\$1}' | sort) echo "NVME disks are: \$NVME_DISK_LIST" | tee -a /home/ubuntu/nvme-initializer.log # there may be 1 or 2 NVME disks, then we split (or not) the mounts between GitLab custom cache and Docker data export NVME_GITLAB=\$(echo "\$NVME_DISK_LIST" | head -n 1) export NVME_DOCKER=\$(echo "\$NVME_DISK_LIST" | tail -n 1) echo "NVME_GITLAB=\$NVME_GITLAB and NVME_DOCKER=\$NVME_DOCKER" | tee -a /home/ubuntu/nvme-initializer.log # format disks if not sudo mkfs -t xfs /dev/\$NVME_GITLAB | tee -a /home/ubuntu/nvme-initializer.log || echo "\$NVME_GITLAB already formatted" # this may already be done sudo mkfs -t xfs /dev/\$NVME_DOCKER | tee -a /home/ubuntu/nvme-initializer.log || echo "\$NVME_DOCKER already formatted" # disk may be the same, then already formated by previous command # mount on /gitlab-host/ and /var/lib/docker/ sudo mkdir -p /gitlab sudo mount /dev/\$NVME_GITLAB /gitlab | tee -a /home/ubuntu/nvme-initializer.log sudo mkdir -p /gitlab/custom-cache sudo mkdir -p /var/lib/docker sudo mount /dev/\$NVME_DOCKER /var/lib/docker | tee -a /home/ubuntu/nvme-initializer.log ### reinstall Docker (which data may have been wiped out) # docker (re)install sudo apt-get -y reinstall docker-ce docker-ce-cli containerd.io docker-compose-plugin | tee -a /home/ubuntu/nvme-initializer.log echo "NVME initialization succesful" | tee -a /home/ubuntu/nvme-initializer.log EOF # set NVME initializer script as startup script sudo tee /etc/systemd/system/nvme-initializer.service >/dev/null <

GitLab team member here, thanks for sharing.

> Still not a big fan of how stiff Yaml pipelines feel in Gitlab CI

Maybe the pipeline editor in "Build > Pipeline editor" can help with live linting, or more advanced features such as parent-child pipelines or merge trains.

If you need tips for optimizing the CI/CD pipeline, suggest following these tips in the docs https://docs.gitlab.com/ee/ci/pipelines/pipeline_efficiency.... or a few more tips in my recent talk "Efficient DevSecOps pipelines in cloud-native world", slides from Chemnitz Linux Days 2023 in https://docs.google.com/presentation/d/1_kyGo_cWi5dKyxi3BfYj...

> and that tickets for what seems like a simple feature [1] hang around for years, but it is nice.

Thanks for sharing. (FYI for everyone) The linked issue suggests a Docker cache cleanup script, which might be helpful. https://gitlab.com/gitlab-org/gitlab-runner/-/issues/27332#n... -> https://docs.gitlab.com/runner/executors/docker.html#clear-t...

If only competitors could do better...

https://gitlab.com/gitlab-org/gitlab-runner/-/issues/2797

GitLab MR for the CACHE_COMPRESSION_LEVEL implementation

And in general just making caching stuff easier. I feel like it is unnecessarily complicated for example to cache apt-get in Gitlab which I assume makes most people not do it.

https://gitlab.com/gitlab-org/gitlab-runner/-/issues/991#not...