Apache Pulsar VS redpanda

Ingest real-time data from Kafka, Pulsar, or CDC sources like Postgresand MySQL, with built-in support for Debezium.

Apache Pulsar: Pulsar is a distributed messaging platform developed under the Apache Foundation. Its notable features include extremely low latency, support for multi-tenancy, geo-replication across regions, and tiered storage capabilities.

From Apache, there's Pulsar, a distributed pub-sub messaging platform with a flexible messaging model, and an intuitive client API. The latest version brings enhanced Key_Shared subscription implementation, secure Docker runtime based on Alpine Linux and Java 21, rate limiting, enhanced client compatibility, and more. Check out the release notes to read more about all the changes since the last release.

Apache Pulsar is an open-source distributed messaging platform originally developed by Yahoo! It provides a highly scalable solution for messaging and stream processing with robust durability and fault tolerance.

Stream-processing platforms such as Apache Kafka, Apache Pulsar, or Redpanda are specifically engineered to foster event-driven communication in a distributed system and they can be a great choice for developing loosely coupled applications. Stream processing platforms analyze data in motion, offering near-zero latency advantages. For example, consider an alert system for monitoring factory equipment. If a machine's temperature exceeds a certain threshold, a streaming platform can instantly trigger an alert and engineers do timely maintenance.

Edit: Here's a good example of what I'm looking for: https://github.com/apache/pulsar. It is a full application that happens to be deployed (or deployable) with Terraform, and the configuration files are available.

In recent years, it has become apparent that almost no production system is complete without real-time data. This can also be observed through the rise of streaming platforms such as Apache Kafka, Apache Pulsar, Redpanda, and RabbitMQ.

It's colored black on the refreshed Apache Pulsar site. https://pulsar.apache.org/

Redpanda: Redpanda is a straightforward, high-throughput, and cost-efficient messaging service. It is implemented in C++, compatible with Kafka, and claims to be 6 times more cost-effective than Kafka while achieving 10 times faster performance.

References [1] Redpanda Console: https://redpanda.com/redpanda-console-kafka-ui [2] Redpanda: https://redpanda.com/ [3] Cluster Deployment of AutoMQ: https://docs.automq.com/zh/docs/automq-opensource/IyXrw3lHriVPdQkQLDvcPGQdnNh [4] Quick Start: https://github.com/redpanda-data/console?tab=readme-ov-file#quick-start [5] Release Redpanda Console: https://github.com/redpanda-data/console/releases/tag/v2.6.0 [6] Redpanda Console Configuration: https://docs.redpanda.com/current/reference/console/config/#example-redpanda-console-configuration-file [7] Kafdrop Github: https://github.com/obsidiandynamics/kafdrop

To learn more about Redpanda and stay up-to-date, see Redpanda's source codes available on GitHub and join the Redpanda Community on Slack with fellow developers and data engineers.

Stream-processing platforms such as Apache Kafka, Apache Pulsar, or Redpanda are specifically engineered to foster event-driven communication in a distributed system and they can be a great choice for developing loosely coupled applications. Stream processing platforms analyze data in motion, offering near-zero latency advantages. For example, consider an alert system for monitoring factory equipment. If a machine's temperature exceeds a certain threshold, a streaming platform can instantly trigger an alert and engineers do timely maintenance.

Yeah it’s just the stack switching itself that is a handful of cycles, but there is not much more overhead for the full VM switch if you structure your embedding the right way. Code the code is source available if you want to peek at it!

https://github.com/redpanda-data/redpanda/blob/dev/src/v/was...

that's a littlebit of a stretch. when you say "no shortage" - outside of redpanda what product exists that actually compete in all deployment modes?

it's a misconception that redpanda is simply a better kafka. the way to think about it is that is a new storage engine, from scratch, that speaks the kafka protocol. similar to all of the pgsql companies in a different space, i.e.: big table pgsql support is not a better postgres, fundamentally different tech. you can read the src and design here: https://github.com/redpanda-data/redpanda. or an electric car is not the same as a combustion engine, but only similar in that they are cars that take you from point a to point b.

Redpanda with the MongoDB Debezium Connector installed. We use Redpanda as a Kafka broker.

And Kafka + Kafka Connect has https://www.confluent.io/ https://aiven.io/ https://upstash.com/ (and not quite Kafka, but protocol-compatible, https://redpanda.com/)