RabbitMQ VS kubernetes

If you've ever worked as an enterprise developer in any moderately complex company, you've likely encountered distributed systems of the kind I want to talk about in this post—two or more systems communicating together via a message queue (MQ), such as RabbitMQ or Apache Kafka. Distributed, message-based systems are ubiquitous in today's programming landscape, especially due to the (now hopefully at least somewhat tempered) microservice architecture frenzy that swept over our field during the past decade.

RabbitMQ - Feature-rich message broker

Alternatives to: Redis (Queue, Pub/Sub), ZeroMQ, RabbitMQ, Apache Kafka, Amazon Simple Queue Service, Google Cloud Pub/Sub

When something important happens (like a new user signing up), that part of the app sends a message—think of it like dropping a letter in a mailbox. Other parts of the app can listen for those messages and react when they arrive. The middleman that passes the message along is called a message broker—and one of the most popular is RabbitMQ. This setup means the sender and the receiver don’t have to be active at the same time. One service can keep moving, and the rest will catch up.

RabbitMQ Official Documentation

rabbitmq

Asynchronous Communication Patterns: Utilize message queues and event-driven architectures to decouple services and improve scalability. By using these patterns, you can handle high volumes of asynchronous requests more efficiently. For example, implementing a message queue like RabbitMQ or Kafka can help manage the flow of requests between services.

Consume open-source queuing services – customers can deploy message brokers such as ActiveMQ or RabbitMQ, to develop asynchronous applications, and when moving to the public cloud, use the cloud providers managed services alternatives.

We are using redis as our broker. You can opt for RabbitMQ which is supported out-of-box by celery.

The solution to this problem started with setting up different teams for both - and ClickOps was coined. As cloud technologies evolved, people realized that it was getting increasingly difficult to keep systems in sync given the room for human error. Naturally, it evolved to the adoption of scripting based pipelines, and it led to the birth of DevOps. This bridged the gap between development and operations quite a bit, and was based on the idea of you build it, you run it, which inadvertently caused shrinkage of operations teams. It was progress alright, but still left techies yearning for something more. Application developers were getting overloaded with the cognitive load of managing infrastructure, and it was getting in the way of product development. Besides this, the complexity of application development was shifting from monoliths to microservices. Upholding the need for separation of concern, technologies evolved and containerization paved the path for taking things to the next level. Docker became somewhat ubiquitous and with the advent of containers, came the need for container orchestration. Google looked over their shoulders and donated Borg to the world, which came to be better known as Kubernetes or k8s. This is where things started to get really spicy, and the cognitive load on developers quickly started to multiply exponentially.

Kubernetes.io - The official project website containing comprehensive documentation, tutorials, and release information. Essential reading for understanding core concepts and staying current with platform updates.

When I need a Dockerfile or Kubernetes manifest, I just describe it to Forge. For instance, I asked Forge to fix a failing Docker build with a permission error, and it immediately spotted that files were being created as root and suggested adding a chown or switching to a non-root user – exactly the real fix we needed. Beyond fixes, Forge can draft new container files from a prompt (“generate a Dockerfile for a Node.js app”), including the right base image and commands. The same goes for K8s: ask it for a deployment YAML for your service, and it will write a working template. This turbocharges our container workflows by automating boilerplate and catching common mistakes.

Kubernetes version 1.33 introduced several new features and bug fixes, and AWS is excited to announce that you can now use Amazon Elastic Kubernetes Service (EKS) and Amazon EKS Distro to run Kubernetes version 1.33. Starting today, you can create new EKS clusters using version 1.33 and upgrade existing clusters to version 1.33 using the EKS console, the eksctl command line interface, or through an infrastructure-as-code tool. Kubernetes version 1.33 includes stable support for sidecar containers, topology-aware routing and traffic distribution, and consideration of taints and tolerations when calculating pod topology spread constraints, ensuring that pods are distributed across different topologies according to their specified tolerance. This release also adds support for user namespaces within Linux pods, dynamic resource allocation for network interfaces, and in-place resource resizing for vertical scaling of pods. To learn more about the changes in Kubernetes version 1.33, see our documentation and the Kubernetes project release notes.

3. Kubernetes (kubernetes/kubernetes) – Container Orchestration at Scale

Self-Healing Pods/Containers: Platforms like Kubernetes inherently offer self-healing capabilities, automatically restarting or rescheduling unhealthy containers or pods to maintain desired service levels. This is fundamental to cloud-native resilience.

Kubernetes Kubernetes is a tool for orchestrating(managing) docker containers. With this tool you can deploy, scale and manage your containerized apps. Kubernetes commonly used in developing and production.

Global settings are easy to check and verify and modern frameworks handle it for you, because there's all these knobs. (I agree it's waaay too many, but that's because there's a runtime and on top of that there's a process manager, and on top of that there's nginx/apache or other reverse proxy.)

Sure, someone can write a Go library to wrap every low-level function to make sure there's some error handling, maybe with closures and generics it would be quite okay.

In Go if you don't see the error handled you know it's not handled anywhere else. Great? Well, sure .. um, maybe? After all if you want to handle it you need to add error handling there. Consequently your code now is 3x as many lines and ~66% of it is returning errors upward. It's the new Assembly.

(I don't think try-catch is good, I think that PHP's error handling is better despite try-catch.)

Porting to Go (for reliability or otherwise), why? There are other languages out there! Especially if you spent the last decade learning about compile-time checks.

I know that k8s (and tons of now-critical software) is written in Go, and it's not a pretty sight -- and instead of having better abstractions there's NASA-cargo-culting[0]. Linux is written in C. It does not make C a great choice for many reasons. (Go is definitely a better choice than C when it comes to memory safety for example, but I prefer Scala or Rust.)

Facebook added their own typing to PHP (and tellingly called it Hack, of course).

[0] https://github.com/kubernetes/kubernetes/blob/ec2e767e593953...