Our great sponsors
-
parallel-consumer
Parallel Apache Kafka client wrapper with per message ACK, client side queueing, a simpler consumer/producer API with key concurrency and extendable non-blocking IO processing.
-
WorkOS
The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
So we want to maximize parallelism while keeping DLQ style error handling and stability. We want to send messages to DLQs if any error happens and we don't want any message to be unprocessed and still be committed. I know there are Confluent Parallel Consumer, Greyhound and an article about Multi-Threaded Messaging with Kafka. But I'm not sure if they are ready for production.
NOTE:
The number of mentions on this list indicates mentions on common posts plus user suggested alternatives.
Hence, a higher number means a more popular project.