workshop-pulsar VS Apache Pulsar

Join our beginner-friendly Pulsar workshop on YouTube and learn how to connect Pulsar with Cassandra for streaming! In this post, we’ll set the scene with an introduction to Pulsar and guide you through four hands-on exercises where you’ll use these free, cloud-native technologies: Katacoda, Kesque, GitPod, and DataStax Astra DB. Each exercise will also be linked to the step-by-step instructions on the DataStax Developers GitHub wiki.

RisingWave is an open-source distributed SQL database for stream processing. RisingWave accepts data from sources like Apache Kafka, Apache Pulsar, Amazon Kinesis, Redpanda, and databases via native Change data capture connections to MySQL and PostgreSQL sources. It uses the concept of materialized view that involves caching the outcome of your query operations and it is quite efficient for long-running stream processing queries.

You can ingest data from different data sources such as message brokers Kafka, Redpanda, Kinesis, Pulsar, or databases MySQL or PostgreSQL using their Change Data Capture (CDC) which is the process of identifying and capturing data changes.

Pulsar Shell (https://github.com/apache/pulsar/issues/16250)

Dynamically scaling your containers is getting easier every day when using something like KEDA. I use Pulsar and there's a Pulsar scaler for it in addition to Kafka, Kinesis, Prometheus metrics, etc.

Good luck with your project! Sounds challenging but very interesting, both product and engineering wise. AWS/GCP services would definitely give you a fast start, and future support for scale and processing which is great for mvp or a pilot, and I would probably start with the mentioned services as well, but from my experience and based on your performance needs, definitely around financial use case and especially stocks you will need close to a millisecond latency across your ecosystem which I think is hard to get over aws and gcp offerings. Have you tried Kafka, Pulsar, or Memphis? Learning curve would be harder, but in long term I think they will be better for the performance requirements

Apache Pulsar 2.10.1

Astra Streaming enables developers to build streaming applications on top of an elastically scalable, multi-cloud messaging and event streaming platform powered by Apache Pulsar. DataStax Astra Streaming is currently in beta, and we’ll be releasing a full demo soon. In the meantime, this article will walk you through a short demo that will provide a great starting point for familiarizing yourself with this powerful new streaming service.

this is an interesting name considering there's also apache pulsar, that's similar to kafka

One such technology is Apache Pulsar® — an open-source, distributed messaging and streaming platform that’s easy to deploy, simple to scale, and packed with developer-friendly APIs. So the next question is: how can you stream from Pulsar to Apache Cassandra®, the powerful NoSQL database designed to support data-heavy applications in the cloud?