Aeron
Apache Pulsar
Our great sponsors
Aeron | Apache Pulsar | |
---|---|---|
20 | 30 | |
7,054 | 13,744 | |
1.1% | 1.2% | |
9.8 | 9.8 | |
2 days ago | 1 day ago | |
Java | Java | |
Apache License 2.0 | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
Aeron
-
LMAX Disruptor – High Performance Inter-Thread Messaging Library
Semi-related is the Aeron project: https://github.com/real-logic/aeron
-
Nálatok mi a helyzet?
- ez itt most egy izgalmasabb product (trading/matching engine, low latency code, aeron alapokon)
-
How do you do UDP Flow control?
Look into Aeron for examples of high performance UDP message sending. We use it for high performance audio messaging, and I previously used it in high frequency trading https://github.com/real-logic/aeron. It is written in Java/C, but the general concepts of back pressure and reliable delivery over UDP are well documented.
- Aeron: Efficient reliable UDP unicast, UDP multicast, and IPC message transport
-
Experience taking the training offer from real-logic Aeron framework creators?
They mention their training offer on the Aeron GitHub page here: https://github.com/real-logic/aeron
-
Low Latency C++ programs for High Frequency Trading (HFT)
Yup the Disruptor paper actually shocked the industry a bit, b/c it was so out of place. BTW, Martin Thompson went on improving the Disruptor, and the result is the Aeron Protocol: https://github.com/real-logic/aeron
- What network messaging library do you recommend?
-
Possibly stupid question, is java the right language for low latency and high throughput web servers?
I was about to suggest Chronicle, but it looks like they have gone closed-source. The older version is still interesting to look through though. Aeron / Disruptor / SBE are good projects for inspiration as well.
-
Looking similar framework with Aeron ( Java) to do benchmark test
We are using this Java Aeron (https://github.com/real-logic/aeron) to build our production distributed messaging cluster. As a Rust lover, Is there any similar lib or framework in our ecosystem to test benchmark with it?
-
if you had to restart at 0 knowledge what would you do?
Java: In the past years C++ in finance has been rapidly supplanted by Java thanks to breakthrough technologies in the past decade like LMAX Disruptor, Chronicle Queue, Azul JVM, and Aeron (not the ergonomic chair, but this one, the transport protocol that breaks kafka performance records out of the park - not really a full kafka replacement, as Kafka enforces subscriber GD and aeron is more of an OSI layer 4 better than TCP; google "Best-effort delivery vs reliable delivery"). There's plenty more but thanks to these technologies, they allowed a Java based stack to perform the latency and throughput requirements needed for high frequency trading/HFT. From top trading firms like Two Sigma to the New York Stock Exchange, they're in Java. For banks, large modern western banks worth their salt and have modernized their systems are dominated by Java, especially thanks to Azul. To list a few banks, ING, Wells Fargo, Credit Suisse, and Barclays are all in Azul. Even at work Java still dominates.
Apache Pulsar
-
Choosing Between a Streaming Database and a Stream Processing Framework in Python
Stream-processing platforms such as Apache Kafka, Apache Pulsar, or Redpanda are specifically engineered to foster event-driven communication in a distributed system and they can be a great choice for developing loosely coupled applications. Stream processing platforms analyze data in motion, offering near-zero latency advantages. For example, consider an alert system for monitoring factory equipment. If a machine's temperature exceeds a certain threshold, a streaming platform can instantly trigger an alert and engineers do timely maintenance.
-
Apache Pulsar VS quix-streams - a user suggested alternative
2 projects | 7 Dec 2023
-
Help finding open source Terraform configurations that are not educational projects or developer tools
Edit: Here's a good example of what I'm looking for: https://github.com/apache/pulsar. It is a full application that happens to be deployed (or deployable) with Terraform, and the configuration files are available.
-
Kafka Is Dead, Long Live Kafka
I am the founder of RisingWave (http://risingwave.com/), an open-source SQL streaming database. I am happy to see the launch of Warpstream! I just reviewed the project and here's my personal opinion:
* Apache Kafka is undoubtedly the leading product in the streaming platform space. It offers a simple yet effective API that has become the golden standard. All streaming/messaging vendors need to adhere to Kafka protocol.
* The original Kafka only used local storage to store data, which can be extremely expensive if the data volume is large. That's why many people are advocating for the development of Kafka Tiered Storage (KIP-405: https://cwiki.apache.org/confluence/display/KAFKA/KIP-405%3A...). To my best knowledge, there are at least five vendors selling Kafka or Kafka-compatible products with tiered storage support:
-- Confluent, which builds Kora, the 10X Kafka engine: https://www.confluent.io/10x-apache-kafka/;
-- Aiven, the open-source tiered storage Kafka (source code: https://github.com/Aiven-Open/tiered-storage-for-apache-kafk...
-- Redpanda Data, which cuts your TCO by 6X (https://redpanda.com/platform-tco);
-- DataStax, which commercializes Apache Pulsar (https://pulsar.apache.org/);
-- StreamNative, which commercializes Apache Pulsar (https://pulsar.apache.org/).
* WarpStream claims to be "built directly on top of S3," which I believe is a very aggressive approach that has the potential to drastically reduce costs, even compared to tiered storage. The potential tradeoff is system performance, especially in terms of latency. As new technology, WarpStream brings novelty, and definitely it also needs to convince users that the service is robust and reliable.
* BYOC (Bring Your Own Cloud) is becoming the default option. Most of the vendors listed above offer BYOC, where data is stored in customers' cloud accounts, addressing concerns about data privacy and security.
I believe WarpStream is new technology to this market, and and would encourage the team to publish some detailed numbers to confirm its performance and efficiency!
-
Analyzing Real-Time Movie Reviews With Redpanda and Memgraph
In recent years, it has become apparent that almost no production system is complete without real-time data. This can also be observed through the rise of streaming platforms such as Apache Kafka, Apache Pulsar, Redpanda, and RabbitMQ.
-
There are about Pulsar 10k users in Slack, but about 70 in this subreddit.
It's colored black on the refreshed Apache Pulsar site. https://pulsar.apache.org/
- Is anyone frustrated with anything about Prometheus?
- Kafka alternatives
-
Is Redpanda going to replace Apache Kafka?
So many tools out there, its just which one do you like, I guess. I like Kafka. Works for our environment and we have a few clusters. People have brought up Cribl to replace our kafka (havent really looked into Cribl and we also run NiFi). I have even heard https://pulsar.apache.org/ , which seems to be almost another flavor of Kafka.
-
Querying microservices in real-time with materialized views
RisingWave is an open-source streaming database that has built-in fully-managed CDC source connectors for various databases, also it can collect data from other sources such Kafka, Pulsar, Kinesis, or Redpanda and it allows you to query real-time streams using SQL. You can get a materialized view that is always up-to-date.
What are some alternatives?
Apache Kafka - Mirror of Apache Kafka
redpanda - Redpanda is a streaming data platform for developers. Kafka API compatible. 10x faster. No ZooKeeper. No JVM!
Embedded RabbitMQ - A JVM library to use RabbitMQ as an embedded service
Apache ActiveMQ - Mirror of Apache ActiveMQ
Apache ActiveMQ Artemis - Mirror of Apache ActiveMQ Artemis
JeroMQ - Pure Java ZeroMQ
Apache Camel - Apache Camel is an open source integration framework that empowers you to quickly and easily integrate various systems consuming or producing data.
Apache RocketMQ - Apache RocketMQ is a cloud native messaging and streaming platform, making it simple to build event-driven applications.
Disruptor-cpp - Port of LMAX Disruptor to C++
RocketMQ