Processing billions of events in real time at Twitter

• Apache Pulsar

  30 13,772 9.8 Java

  Apache Pulsar - distributed pub-sub messaging system

  

This is basically an ad for GCP right?

That said, it looks like Kafka is by far and away the way to handle persistent logs/events at scale. AFAIK a company here in Japan called LINE has all their messaging flowing through a large kafka cluster themselves.

Wonder if anyone is running large NATS Jetstream[0]/Liftbridge[1] or Pulsar[2] (yahoo runs those) clusters. I guess Pulsar might be #2 in terms of adoption at large scale?

[0]: https://docs.nats.io/jetstream/jetstream

[1]: https://liftbridge.io/

[2]: https://pulsar.apache.org/

• liftbridge

  10 2,535 6.3 Go

  Lightweight, fault-tolerant message streams.

  

This is basically an ad for GCP right?

That said, it looks like Kafka is by far and away the way to handle persistent logs/events at scale. AFAIK a company here in Japan called LINE has all their messaging flowing through a large kafka cluster themselves.

Wonder if anyone is running large NATS Jetstream[0]/Liftbridge[1] or Pulsar[2] (yahoo runs those) clusters. I guess Pulsar might be #2 in terms of adoption at large scale?

[0]: https://docs.nats.io/jetstream/jetstream

[1]: https://liftbridge.io/

[2]: https://pulsar.apache.org/

• InfluxDB

  www.influxdata.com featured

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• NATS

  106 14,816 9.8 Go

  High-Performance server for NATS.io, the cloud and edge native messaging system.

  

NATS is built with golang which is one of the reasons I like it…

https://github.com/nats-io/nats-server

Suggest a related project