Apache Kafka
redpanda
Our great sponsors
Apache Kafka | redpanda | |
---|---|---|
26 | 69 | |
27,275 | 8,734 | |
1.3% | 2.8% | |
9.9 | 10.0 | |
about 22 hours ago | 4 days ago | |
Java | C++ | |
Apache License 2.0 | - |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
Apache Kafka
-
On Implementation of Distributed Protocols
Apache Kafka — a distributed event streaming platform implementing a variant of the Raft consensus protocol (written in Java, integrated with Scala);
- Implementing tagged fields for Kafka Protocol
-
Help me identify this design pattern
Spring does this during autoconfiguration. For example this and this. When the user adds a configuration then it gets to overwrite the default from the template. I am looking for something similar, perhaps simpler approach.
- Kafka Broker Config properties
- Scala DevInTraining looking to contribute to projects
- *bip*
-
What is Kafka ?
Source and documentation on GitHub
-
A simple file source/sink connector?
Code is still in trunk though. https://github.com/apache/kafka/tree/trunk/connect/file/src/main/java/org/apache/kafka/connect/file
-
Can someone please eli5 how the hierarchical timing wheel algorithm works?
I briefly described the algorithm in this article and there is a wonderful article from Kafka that goes into more depth in their general purpose implementation. My implementation is specialized and over optimized in comparison, e.g. by using bit manipulation to avoid more expensive division/modulus instructions. Tokio rewrote their timerwheel after I showed them mine, borrowing some ideas but also staying more general. Hope that helps!
-
How-to-Guide: Contributing to Open Source
Apache Kafka
redpanda
-
Choosing Between a Streaming Database and a Stream Processing Framework in Python
Stream-processing platforms such as Apache Kafka, Apache Pulsar, or Redpanda are specifically engineered to foster event-driven communication in a distributed system and they can be a great choice for developing loosely coupled applications. Stream processing platforms analyze data in motion, offering near-zero latency advantages. For example, consider an alert system for monitoring factory equipment. If a machine's temperature exceeds a certain threshold, a streaming platform can instantly trigger an alert and engineers do timely maintenance.
-
The best WebAssembly runtime may be no runtime at all
Yeah it’s just the stack switching itself that is a handful of cycles, but there is not much more overhead for the full VM switch if you structure your embedding the right way. Code the code is source available if you want to peek at it!
https://github.com/redpanda-data/redpanda/blob/dev/src/v/was...
-
redpanda VS quix-streams - a user suggested alternative
2 projects | 7 Dec 2023
-
Kafka Is Dead, Long Live Kafka
that's a littlebit of a stretch. when you say "no shortage" - outside of redpanda what product exists that actually compete in all deployment modes?
it's a misconception that redpanda is simply a better kafka. the way to think about it is that is a new storage engine, from scratch, that speaks the kafka protocol. similar to all of the pgsql companies in a different space, i.e.: big table pgsql support is not a better postgres, fundamentally different tech. you can read the src and design here: https://github.com/redpanda-data/redpanda. or an electric car is not the same as a combustion engine, but only similar in that they are cars that take you from point a to point b.
-
Real-time Data Processing Pipeline With MongoDB, Kafka, Debezium And RisingWave
Redpanda with the MongoDB Debezium Connector installed. We use Redpanda as a Kafka broker.
- Redpanda
-
Flink CDC / alternatives
And Kafka + Kafka Connect has https://www.confluent.io/ https://aiven.io/ https://upstash.com/ (and not quite Kafka, but protocol-compatible, https://redpanda.com/)
-
The Redpanda Project
There exists a C++ project which was created after Rust the language was available. github.com/redpanda-data/redpanda/
-
SOCKS Proxy Server Architecture for High Concurrency
I suggest you check out io_uring and thread per core architecture. Applications like scylladb and redpanda have thread per core architecture and use io_uring for async io.
-
Kafka alternatives
Redpanda
What are some alternatives?
celery - Distributed Task Queue (development branch)
Apache Pulsar - Apache Pulsar - distributed pub-sub messaging system
Apache ActiveMQ Artemis - Mirror of Apache ActiveMQ Artemis
NATS - High-Performance server for NATS.io, the cloud and edge native messaging system.
jetstream - JetStream Utilities
Aeron - Efficient reliable UDP unicast, UDP multicast, and IPC message transport
RabbitMQ - Open source RabbitMQ: core server and tier 1 (built-in) plugins
kafkacat - Generic command line non-JVM Apache Kafka producer and consumer [Moved to: https://github.com/edenhill/kcat]
Apache Qpid - Mirror of Apache Qpid
Seastar - High performance server-side application framework