kubernetes VS Apache ZooKeeper

Cloud-Native Friendly: Lightweight and fast, Go apps fit perfectly into containerized environments like Docker and Kubernetes.

Over the years, Indian developers have played increasingly vital roles in many international projects. From contributions to frameworks such as Kubernetes and Apache Hadoop to the emergence of homegrown platforms like OpenStack India, India has steadily carved out a global reputation as a powerhouse of open source talent.

I'd also strongly recommend this view of how Kubernetes uses cgroups, showing similar drill downs for how everything gets managed.

I've been a bit apoplectic in the past that cgroups seemed not super helpful in Kubernetes, but this really showed me how the different Kubernetes QoS levels are driven by similar juggling of different cgroups.

I'm not sure if this makes use of cpu.max.burst or not. There's a fun article that monkeys with these cgroups directly, which is neat to see. It also links to an ask that Linux support the new (5.14) CFS Burst system. Which is a whole nother fun rabbit hole to go down! https://medium.com/@christian.cadieux/kubernetes-throttling-... https://github.com/kubernetes/kubernetes/issues/104516

Kubernetes isn't just for container orchestration—it packs a powerful built-in service discovery system that's changing how developers think about service connectivity. It uses DNS under the hood, along with environment variables, to help services find each other.

https://github.com/kubernetes/kubernetes/commit/7fef0a4f6a44...

Of course, we'd already fixed other issues like Kubelet listening on a secondary debug port with no authentication. Those problems stemmed from its origins as a make-it-possible hacker project and it took a while to pivot it to something usable in an enterprise.

For a comprehensive overview, explore the Kubernetes 1.33 release notes and GitHub changelog. Engage with the community at events like KubeCon or join the Kubernetes Slack to collaborate on the future of cloud-native computing. With Octarine, Kubernetes continues to shine as the backbone of modern infrastructure.

Historical Note: For a brief period around Kubernetes 1.10, the default was accidentally changed to HostToContainer before being reverted. If you're managing a very old cluster, be aware of this possibility (#62462).

Imagine trying to keep a fleet of ships sailing smoothly across the ocean. You need to ensure each ship has enough crew, fuel, and cargo, and that they're all heading in the right direction. This is a complex task, requiring constant monitoring and adjustments. In the world of Kubernetes, Controllers and Operators play a similar role, ensuring your applications run smoothly and efficiently. This blog post delves deep into the differences between these two essential components, providing clear examples to illustrate their functionalities and use cases.

Kubernetes has become the de facto standard for container orchestration. With the rise of microservices and cloud-native applications, managing network traffic within a Kubernetes cluster has become increasingly critical. The Ingress API has been the traditional solution for managing external access to services in Kubernetes. However, with the evolution of Kubernetes and the need for more advanced traffic management capabilities, the Gateway API has emerged as a powerful alternative. This post provides a detailed comparison of Gateway API and Ingress API, highlighting the advantages of Gateway API and offering recommendations for when to choose it over Ingress API.

Kubernetes is an innovative open-source platform designed for container orchestration. It automates critical processes such as deploying, scaling, and managing containerised applications. Originally developed by Google, Kubernetes is now watched by the Cloud Native Computing Foundation. In essence, Kubernetes serves as a powerful framework for managing your applications, enabling seamless deployment and scalability tailored to your needs.

Zookeeper is a distributed coordination service used in older versions of Kafka to manage cluster metadata, leader election, and configuration. It ensures consistency and synchronization across Kafka brokers.

Using a service discovery mechanism: A service discovery mechanism, such as etcd or ZooKeeper, can help to manage the complexity of microservices by providing a centralized registry of available services and their instances.

Apache ZooKeeper: A centralized service to generate unique IDs and prevent race conditions between instances.

Apache ZooKeeper — a distributed coordination, synchronization, and configuration service (written in Java);

To use Kafka, we also need to deploy a service that keeps configuration informations such as Zookeeper.

Failure Detection and Recovery It’s not enough to have backup systems. It’s also crucial to detect failures quickly. Modern systems employ monitoring tools and rely on distributed coordination systems such as Zookeeper or etcd to identify faults in real-time: once detected, recovery mechanisms are triggered to restore the service.

zookeeper: is (was?) used for secrets management. it was also used as a basic health check, but has been since been replaced.

You can install Kafka from https://kafka.apache.org/quickstart. Because Druid and Kafka both use Apache Zookeeper, I opted to use the Zookeeper deployment that comes with Druid, so didn’t start it with Kafka. Once running, I created two topics for me to post the data into, and for Druid to ingest from:

Please note that we use Zookeeper as the Governance Center.