What I Wish Someone Would Have Told Me About Using Rabbitmq

• liftbridge

  10 2,532 6.4 Go

  Lightweight, fault-tolerant message streams.

  

• redpanda

  69 8,784 10.0 C++

  Redpanda is a streaming data platform for developers. Kafka API compatible. 10x faster. No ZooKeeper. No JVM!

  

• InfluxDB

  www.influxdata.com sponsored

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• Apache Pulsar

  30 13,744 9.8 Java

  Apache Pulsar - distributed pub-sub messaging system

  

• RabbitMQ

  92 11,590 10.0 Starlark

  Open source RabbitMQ: core server and tier 1 (built-in) plugins

  

Here is what I dislike about RabbitMQ: by default it opens several ports on external interfaces and doesn't provide an easy way to bind them to localhost only. To be specific, those are epmd port 4369, and RabbitMQ's port 5672, 15672, 25672. With help of Google I managed to move three of those ports to localhost, however the cluster communications port 25672 cannot be bound to localhost only. I found an issue on Github and developers' point is that it doesn't make sense to build a cluster from a single node, so you must have this port exposed to whole Internet so that any random stranger can connect to your RabbitMQ instance [1].

So it seems that RabbitMQ wants to advertise how scalable it is by enabling clustering by default and accepting connections from anyone even though it is not secure. It is targeted at large corporations with giant clusters and doesn't care about developers who just need a single instance. Despite my guess that most developers actually do not have a volume of messages that would justify setting up a cluster.

And as RabbitMQ is written in an exotic language (Erlang) I cannot even read the code.

This is the problem not only with RabbitMQ, Elasticsearch also has clustering settings enabled by default, so when you try to run two independent instances, they connect to each other and start exchanging data and creating problems without you expecting it. So annoying and so difficult to disable. In earlier versions, as I remember, they would also try to scan local network and connect to any node they could find.

[1] https://github.com/rabbitmq/rabbitmq-server/issues/1661

• debezium

  80 9,857 9.9 Java

  Change data capture for a variety of databases. Please log issues at https://issues.redhat.com/browse/DBZ.

  

In the first iteration of using confirms, we did not have the outbox but only logged how long it took to get the confirmation. After 3 seconds, we would throw out the expected confirmation. If a confirmation took longer than that, we would log that we received an unknown confirmation.

We hoped it would be fast enough that we can just wait for the confirmation before committing the transaction.

The official documentation says

> This means that under a constant load, latency for basic.ack can reach a few hundred milliseconds

I never did statistics, just looked at the log file. IIRC most were acceptable but > 3s occurred frequently enough that we abandoned that plan.

We considered using Debezium[0], but decided on the current solution as it could be solved entirely with the current services and infrastructure whereas Debezium would have required us to deploy Kafka, Zookeeper, the connector service.

[0] https://debezium.io/

• WorkOS

  workos.com sponsored

  The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

  

Suggest a related project