Our great sponsors
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
gl-infra
- Incident Review for Site-Wide Outage for Gitlab.com – Stale Terraform Pipeline
- Gitlab Friday (July 7) Outage Incident Review
- Gitlab Outage 5 Whys
- GitLab.COM down?
-
Gitlab.com Is Completely Down
GitLab team member here. Thanks for asking.
Incidents can have different types, i.e. when an application bug or performance regression is discovered, this can involve reverting MRs and rolling back releases. The Platform, Delivery group has a top-level responsibility for ensuring continuous delivery of the GitLab application software to GitLab SaaS, https://about.gitlab.com/handbook/engineering/infrastructure...
Other incidents may involve hardware or infrastructure failures, or a combination of both, infrastructure failure that renders GitLab application services unavailable. This requires cross-functional collaboration from infrastructure, product, engineering, etc. teams in the incident.
To get a better understanding here, it is helpful to review the incident management handbook https://about.gitlab.com/handbook/engineering/infrastructure...
Additional helpful information:
- The GitLab.com SaaS production architecture is documented in https://about.gitlab.com/handbook/engineering/infrastructure...
- The Monitoring of GitLab.com handbook provides insights into monitoring workflows, incident management, SLAs, etc. https://about.gitlab.com/handbook/engineering/monitoring/
- Runbooks https://about.gitlab.com/handbook/engineering/infrastructure...
For the current incident discussed in this HN thread, the review issue can be followed in https://gitlab.com/gitlab-com/gl-infra/production/-/issues/1... to learn more.
-
GitLab internal api unreachable
Lol. They let a certificate expire: https://gitlab.com/gitlab-com/gl-infra/production/-/issues/14422
- Is there a security incident ongoing?
- Does Gitlab.com have a security incident?
- fb giresun canli mac izle
- RIZIN LANDMARK 5 in YOYOGI ライブ
runbooks
-
Monitoring system health, Logs and scheduled jobs
There are also good patterns for monitoring the status of jobs.
-
Jenkins build time in Prometheus
Check out this job monitoring pattern.
-
Alert Reference Documentation
For a freely available resource, I found GitLab’s Runvbooks a good example to strive towards. But this is the output of a large team and several years. My advice is to just start with something simple.
-
Gitlab is ALWAYS down
It's a weighted combination of all services. The SLA weightings for all service areas are defined in the gitlab.com service catalog.
- An impatient SRE's guide to deleting alerts
-
Any simple ways to record the duration of arbitrary processes?
Take a look at this pushgateway-based job monitoring pattern.
- Gitlab S-1
-
To use Gitlab Dashboards - How to use grafonnet with jsonnet to produce a json file that I can use with Grafana
I want to use the GitLab dashboards here in my local Grafana environment - https://gitlab.com/gitlab-com/runbooks/-/tree/master/dashboards
-
Looking for Grafana dashboards to help us monitor our GitLab services running in Kubernetes
Also, take a look at the code https://gitlab.com/gitlab-com/runbooks
-
Is a runbook what I’m looking for?
GitLab puts everything in a git repo in markdown. A single repo that contains a few different things. All of the production alerts are sourced from there. This makes it possible to review alerts and runbook changes in the same code review.
What are some alternatives?
www-gitlab-com
postgres_exporter - A PostgreSQL metric exporter for Prometheus
thanos-operator - Kubernetes operator for deploying Thanos
operator - Kubernetes operator for Victoria Metrics
gitlab
mermaid - Generation of diagrams like flowcharts or sequence diagrams from text in a similar manner as markdown
git2git - Handy library for copying repositories from one git host to another
diagrams - :art: Diagram as Code for prototyping cloud system architectures
gitlab-foss
govuk-infrastructure - Terraform turnup automation for the EKS Kubernetes clusters that host GOV.UK. See https://github.com/alphagov/govuk-helm-charts for application config.
prio-server - A Prio server implementation.
kube-prometheus - Use Prometheus to monitor Kubernetes and applications running on Kubernetes