Go cloud-native

Open-source Go projects categorized as cloud-native | Edit details

Top 23 Go cloud-native Projects

  • GitHub repo tidb

    TiDB is an open source distributed HTAP database compatible with the MySQL protocol

    Project mention: Comparing Nginx Performance in Bare Metal and Virtual Environments | news.ycombinator.com | 2021-10-29

    I do agree with you in that regard, however, that's also a dangerous line of thinking.

    There are attempts to provide horizontal scalability for RDBMSes in a transparent way, like TiDB https://pingcap.com/ (which is compatible with the MySQL 5.7 drivers), however, the list of functionality that's sacrificed to achieve easily extensible clusters is a long one: https://docs.pingcap.com/tidb/stable/mysql-compatibility

    There are other technologies, like MongoDB, which sometimes are more successful at a clustered configuration, however most of the traditional RDBMSes work best in a leader-follower type of replication scenario, because even those aforementioned systems oftentimes have data consistency issues that may eventually pop up.

    Essentially, my argument is that the lack of good horizontally scalable databases or other data storage solutions is easily explainable by the fact that the problem itself isn't solvable in any easy way, apart from adopting eventual consistency, which is probably going to create more problems than it will solve in case of any pre-existing code that makes assumptions about what ways it'll be able to access data and operate on it: https://en.wikipedia.org/wiki/Fallacies_of_distributed_compu...

    To that end, i'd perhaps like to suggest an alternative: use a single vertically scalable RDBMS instance when possible, with a hot standby if you have the resources for that. Let the architecture around it be horizontally scalable instead, and let it deal with the complexities of balancing the load and dealing with backpressure - introduce a message queue if you must, maybe even an in-memory one for simplicity's sake, or consider an event based architecture where "what needs to be done" is encapsulated within a data structure that can be passed around and applied whenever possible. In my eyes, such solutions can in many cases be better than losing the many benefits of having a single source of truth.

    Alternatively, consider sharding as a possibility, or, alternatively, do some domain driven design, figure out where to draw some boundaries and split your service into multiple ones that cover the domain with which you need to work with. Then you have one DB for sales, one for account management, one for reports and so on, all separated by something as simple as REST interfaces and with rate limits or any of the other mechanisms.

    If, however, neither of those two groups of approaches don't seem to be suitable for the loads that you're dealing with, then you probably have a team of very smart people and a large amount of resources to figure out what will work best.

    To sum up, if there are no good solutions in the space, perhaps that's because the problems themselves haven't been solved yet. Thus, sooner or later, they'll need to be sidestepped and their impact mitigated in whatever capacity is possible.

  • GitHub repo Ory Hydra

    OpenID Certified™ OpenID Connect and OAuth Provider written in Go - cloud native, security-first, open source API security for your infrastructure. SDKs for any language. Works with Hardware Security Modules. Compatible with MITREid.

    Project mention: What type of software do you write at your workplace? | reddit.com/r/golang | 2022-01-17

    For hobby work I've used Gorilla Mux for web APIs and written a UI in Angular and I depended upon Badgerdb for data storage since it's a fantastic in-process datastore and well supported since it's the backend for Dgraph. I'm quite happy with those choices, but I'm going to be rewriting my hobby work and using Postgres by way of Ent, playing with Go HTML templating, and using just the standard library for routing (probably... might go back to Gorilla). I might use ORY Hydra for all my auth needs this time around, but of course, hobby work moves far slower than professional work, so it's hard to say what all I will get around to doing.

  • Scout APM

    Less time debugging, more time building. Scout APM allows you to find and fix performance issues with no hassle. Now with error monitoring and external services monitoring, Scout is a developer's best friend when it comes to application development.

  • GitHub repo NATS

    High-Performance server for NATS.io, the cloud and edge native messaging system.

    Project mention: Distributed messaging with NATS | dev.to | 2022-01-20

    Recently, I was building an application where I wanted to take event driven approach for async communication between the microservices. Usually I would use Apache Kafka, RabbitMQ, Redis Streams or a managed solution like AWS SNS, Google Cloud Pub/Sub but this time I wanted to keep my implementation easy and cost minimal yet not affect my scalability or increase technical debt for the future. After few hops on StackShare, I found NATS as a popular alternative and after trying it out, I really liked it. That evening I migrated everything from Apache Kafka to NATS. So, in this article we'll learn what NATS is and how to get started with it.

  • GitHub repo rook

    Storage Orchestration for Kubernetes

    Project mention: First raspberry pi cluster | reddit.com/r/homelab | 2021-11-24

    I wonder how well rook.io performs with 8GB pis and usb SSDs.

  • GitHub repo kubesphere

    The container platform tailored for Kubernetes multi-cloud, datacenter, and edge management ⎈ 🖥 ☁️

    Project mention: Top 200 Kubernetes Tools for DevOps Engineer Like You | dev.to | 2022-01-15

    Minikube - minikube implements a local Kubernetes cluster KubeSphere - Easy-to-use Production Ready Container Platform https://kubesphere.io skippbox - A Desktop application for k8s kind - Kubernetes IN Docker - local clusters for testing Kubernetes https://kind.sigs.k8s.io/ k3d - k3d is a lightweight wrapper to run k3s (Rancher Lab’s minimal Kubernetes distribution) in docker.

  • GitHub repo conduit

    Ultralight, security-first service mesh for Kubernetes. Main repo for Linkerd 2.x.

    Project mention: Nginx – The Architecture of Open Source Applications | news.ycombinator.com | 2021-11-02

    > As a relatively young dev, the idea of a "web server" as a standalone binary that serves your application (vs a library that you use to write your own "server") feels strange.

    In my eyes, the ideal setup is one that's layered: where you have an ingress that's basically a load balancer that also ensures that you have SSL/TLS certificates, enforces rate limits, perhaps is used for some very basic logging, or can optionally do any URL rewriting that you need. Personally, i think that Caddy (https://caddyserver.com/) is lovely for this, whereas some people prefer something like Traefik (https://traefik.io/), though the older software packages like Nginx (https://nginx.org/en/) or even Apache (https://www.apache.org/) are good too, as long as the pattern itself is in place.

    Then, you may additionally have any sorts of middleware that you need, such as a service mesh for service discovery, or providing internal SSL/TLS - personally Docker Swarm (https://docs.docker.com/engine/swarm/) overlay networks have always been enough for me in this regard, though some people enjoy other solutions, such as Hashicorp Consul (https://www.consul.io/), or maybe something intended for Kubernetes or other platforms that you already may be using, like Linkerd (https://linkerd.io/).

    Finally, you have your actual application with its server. Personally, i think that the web server should be embedded (for example, embedded Tomcat with Spring Boot) or indeed just be a library that's a part of the application executable, as long as you can update it easily enough by rebuilding the application - containers are good for this, but aren't strictly necessary, since sometimes other forms of automation and packaging are also enough.

    The reason why i believe this, is because i've seen plenty of deployments where that just isn't the case:

      - attempts to store certificates within the application, each application server having different requirements for the formats to be used, making management (and automation) of renewal a total nightmare

  • GitHub repo fission

    Fast and Simple Serverless Functions for Kubernetes

    Project mention: Serverless : Exécuter ses containers directement comme des fonctions avec Fission et Rancher RKE2 … | dev.to | 2022-01-04


  • SonarQube

    Static code analysis for 29 languages.. Your projects are multi-language. So is SonarQube analysis. Find Bugs, Vulnerabilities, Security Hotspots, and Code Smells so you can release quality code every time. Get started analyzing your projects today for free.

  • GitHub repo OPA (Open Policy Agent)

    An open source, general-purpose policy engine.

    Project mention: Exploring and Tech – Internationally Awesome with Peter ONeill | dev.to | 2022-01-19

    And so, from there, our maintainers are super active. They are very quick to respond. And so, if you need help understanding what style of coding we do, how we like things to be submitted, we are very active in engaging, especially with first-time contributors. Finding us online, joining the Slack, and just saying hello is that first step.

  • GitHub repo Dragonfly

    Dragonfly is an intelligent P2P based image and file distribution system. (by dragonflyoss)

    Project mention: Can Kubernetes pre-pull and cache images? | reddit.com/r/kubernetes | 2021-07-06
  • GitHub repo juicefs

    JuiceFS is a distributed POSIX file system built on top of Redis and S3.

    Project mention: JuiceFS 1.0-beta1 was released under Apache 2.0 | reddit.com/r/golang | 2022-01-13
  • GitHub repo kubeedge

    Kubernetes Native Edge Computing Framework (project under CNCF)

    Project mention: KubeEdge v1.7 is out now | news.ycombinator.com | 2021-06-01
  • GitHub repo buildkit

    concurrent, cache-efficient, and Dockerfile-agnostic builder toolkit

    Project mention: Gokrazy – A Native Go Userland | news.ycombinator.com | 2022-01-16

    Relatedly, there's also buildkit[1] and that entire ecosystem. I've been meaning to check out the experience of building containers using Nix, but either way it seems very interesting.

    [1]: https://github.com/moby/buildkit

  • GitHub repo crossplane

    Your Universal Control Plane

    Project mention: Helm chart dependency lifecycle management | reddit.com/r/kubernetes | 2022-01-16

    Crossplane is basically a Kubernetes native replacement for Terraform. Functionally similiar to using the TFC operator, but avoids the need for an external tool to manage the infrastructure.

  • GitHub repo pouch

    An Efficient Enterprise-class Container Engine

    Project mention: Ask HN: Any Good Alternative for Docker? | news.ycombinator.com | 2021-08-31
  • GitHub repo chaos-mesh

    A Chaos Engineering Platform for Kubernetes.

    Project mention: Share your #ChaosMeshStory! | reddit.com/r/ChaosEngineering | 2021-12-29

    🐒 Chaos Mesh will turn 2 on 2021.12.31! We're grateful for every contribution that helped this project grow, and we’d like to hear your Chaos Mesh story!

  • GitHub repo easegress

    A Cloud Native traffic orchestration system

    Project mention: Easegress: An all-rounder traffic orchestration system | news.ycombinator.com | 2021-06-16
  • GitHub repo k3sup

    bootstrap Kubernetes with k3s over SSH < 1 min 🚀

    Project mention: Headscale : une implémentation open source et auto-hébergée du serveur de contrôle Tailscale … | dev.to | 2022-01-10

    GitHub - alexellis/k3sup: bootstrap Kubernetes with k3s over SSH < 1 min 🚀

  • GitHub repo NATS

    Golang client for NATS, the cloud native messaging system.

    Project mention: Distributed messaging with NATS | dev.to | 2022-01-20

    Now that our NATS server is running, we'll be using Go and Node.js clients to connect to it for simple demonstration. Not familiar with Go or Node? Don't worry NATS has clients available in over 40 languages!

  • GitHub repo stolon

    PostgreSQL cloud native High Availability and more.

    Project mention: PostgreSQL: Kubegres is available as open source | news.ycombinator.com | 2021-10-05

    I wonder how is this compared to Stolon https://github.com/sorintlab/stolon?

  • GitHub repo gloo

    The Feature-rich, Kubernetes-native, Next-Generation API Gateway Built on Envoy

    Project mention: Releasing Gloo 0.4.0 | reddit.com/r/rust | 2021-10-24

    Does it bug anyone else when people come out promoting their "new" thing that has used the name of an existing well established thing? https://github.com/solo-io/gloo like... how hard is it to search and see the name's already taken?

  • GitHub repo kruise

    Automate application management on Kubernetes (project under CNCF)

    Project mention: Top 200 Kubernetes Tools for DevOps Engineer Like You | dev.to | 2022-01-15

    kops - Production Grade K8s Installation, Upgrades, and Management silver-surfer - Check ApiVersion compatibility and provide Migration path for Kubernetes objects when upgrading Kubernetes to latest versions Kube-ops-view - Kubernetes Operational View - read-only system dashboard for multiple K8s clusters kubeprompt - Kubernetes prompt info Metalk8s - An opinionated Kubernetes distribution with a focus on long-term on-prem deployments kind - Kubernetes IN Docker - local clusters for testing Kubernetes Clusterman - Cluster Autoscaler for Kubernetes and Mesos Cert-manager - Automatically provision and manage TLS certificates Goldilocks - Get your resource requests "Just Right" katafygio - Dump, or continuously backup Kubernetes objets as yaml files in git Rancher - Complete container management platform Sealed Secrets - A Kubernetes controller and tool for one-way encrypted Secrets OpenKruise/Kruise - Automate application workloads management on Kubernetes https://openkruise.io kubectl snapshot - Take Cluster Snapshots kapp - simple deployment tool focused on the concept of "Kubernetes application" — a set of resources with the same label https://get-kapp.io keda - Event-driven autoscaler for Kubernetes Octant - To better understand the complexity of Kubernetes clusters Portainer - Portainer inside a Kubernetes environment Gardener - Deliver fully-managed clusters at scale everywhere with your own Kubernetes-as-a-Service Kubed - Kubernetes Cluster Operator Daemon Kubestack - Kubestack is the free and open-source GitOps framework to codify your custom platform stack using Terraform.

  • GitHub repo devspace

    DevSpace - The Fastest Developer Tool for Kubernetes ⚡ Automate your deployment workflow with DevSpace and develop software directly inside Kubernetes.

    Project mention: apf: Bidirectional port-forwarding for docker and kubernetes | reddit.com/r/kubernetes | 2021-12-30
  • GitHub repo kuma

    🐻 The Universal Service Mesh. CNCF Sandbox Project.

    Project mention: Powering Kubernetes in the Cloud with Kuma Service Mesh | dev.to | 2021-08-16

    Another important change to make is that when you create the cluster, change the Nodes in the "Default pool" to use the COS (not COS_CONTAINERD) image type. There are some underlying issues when using Kuma with GKE, as noted in this GitHub issue, and this is the currently recommended workaround. Otherwise, you will hit pod initializing issues that affect certificate provisioning.

NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020). The latest post mention was on 2022-01-20.

Go cloud-native related posts


What are some of the best open-source cloud-native projects in Go? This list will help you:

Project Stars
1 tidb 30,193
2 Ory Hydra 12,014
3 NATS 10,381
4 rook 9,428
5 kubesphere 8,653
6 conduit 7,984
7 fission 6,720
8 OPA (Open Policy Agent) 6,128
9 Dragonfly 5,732
10 juicefs 4,715
11 kubeedge 4,646
12 buildkit 4,627
13 crossplane 4,517
14 pouch 4,482
15 chaos-mesh 4,363
16 easegress 4,173
17 k3sup 4,133
18 NATS 3,734
19 stolon 3,511
20 gloo 3,258
21 kruise 2,925
22 devspace 2,609
23 kuma 2,549
Find remote jobs at our new job board 99remotejobs.com. There are 29 new remote jobs listed recently.
Are you hiring? Post a new remote job listing for free.
OPS - Build and Run Open Source Unikernels
Quickly and easily build and deploy open source unikernels in tens of seconds. Deploy in any language to any cloud.