confluent-kafka-python
aiokafka
Our great sponsors
| confluent-kafka-python | aiokafka | |
|---|---|---|
| 9 | 3 | |
| 3,632 | 1,049 | |
| 1.8% | 4.0% | |
| 6.3 | 8.6 | |
| 3 days ago | 3 days ago | |
| Python | Python | |
| GNU General Public License v3.0 or later | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
confluent-kafka-python
-
Show HN: Confluent Kafka support added to FastStream v0.4.0rc0
Responding to popular demand, the latest 0.4.0rc0 version introduces support for Kafka stream processing using Confluent Kafka's Python library - https://github.com/confluentinc/confluent-kafka-python.
-
confluent-kafka-python VS quix-streams - a user suggested alternative
2 projects | 7 Dec 2023
-
Recently joined a DE team and I've been asked to study async, multiprocessing, queuing, and Kafka. Can anybody tell me how to proceed and also share resources that I can use.
- confluent kafka library (https://github.com/confluentinc/confluent-kafka-python) uses librdkafka, and is faster that kafka-python - https://www.bytewax.io/(https://github.com/bytewax/bytewax) is another option. Handles the dirty work of exchanging data across processes for parallel stateful processing
-
New to kafka..
Here are examples https://github.com/confluentinc/confluent-kafka-python
-
On Efficiently Partitioning a Topic in Apache Kafka
I just wanted to mention that pykafka is currently unmaintained and archived on GitHub:
https://github.com/Parsely/pykafka
pykafka was originally developed and maintained by my team at Parse.ly, but we no longer maintain it. We instead encourage folks to use confluent-kafka-python, which is what we have ourselves switched to in our production systems:
https://github.com/confluentinc/confluent-kafka-python
(pykafka was developed at a time before Confluent invested in their own Python binding. Some of the history of the project is described in this 2016 blog post[1] and our original 2015 announcement[2].)
[1]: https://blog.parse.ly/pykafka-now/
[2]: https://blog.parse.ly/announcing-pykafka-python-support-for-...
-
Is there a Python API for event-driven Kafka consumer?
confluent-kafka
-
Kafla producer over internet
I also found that there is a library provided by confluent https://github.com/confluentinc/confluent-kafka-python which maybe helpful here. Is that right? I see a section "SSL certificates" but couldnt get much insight out of this.
-
Confluent Kafka Python Schema Registry: Why the consumer does not need it?
ProtobugDeserializer does not allow anything but the protoBuf message type (see here):
-
Learning Apache Kafka
I'd start off with some simple producer/consumer code to get an understanding of interacting with Kafka's APIs from Python - there's a Python client with some good examples of the producer and consumer as well as more complex examples.
aiokafka
-
FastStream: Python's framework for Efficient Message Queue Handling
Our journey with FastStream started when we needed to integrate our machine learning models into a customer's Apache Kafka environment. To streamline this process, we created FastKafka using AIOKafka, AsyncAPI, and asyncio. It was our first step in making message queue management easier.
-
Improving Kafka interfaces
aiokafka - https://github.com/aio-libs/aiokafka
-
Recently joined a DE team and I've been asked to study async, multiprocessing, queuing, and Kafka. Can anybody tell me how to proceed and also share resources that I can use.
It’s hard to understand exactly what is meant by what you posted but could they be referring to asyncio libraries for use with Kafka in this process? For example, https://github.com/aio-libs/aiokafka
What are some alternatives?
Confluent Kafka Golang Client - Confluent's Apache Kafka Golang client
Faust - Python Stream Processing
kafka-python - Python client for Apache Kafka
faust - Python Stream Processing. A Faust fork
quix-streams - A Python library for building containerized ML and Generative AI applications with Apache Kafka.
faststream - FastStream is a powerful and easy-to-use Python framework for building asynchronous services interacting with event streams such as Apache Kafka, RabbitMQ, NATS and Redis.
demo-scene - 👾Scripts and samples to support Confluent Demos and Talks. ⚠️Might be rough around the edges ;-) 👉For automated tutorials and QA'd code, see https://github.com/confluentinc/examples/
Propan - Propan is a powerful and easy-to-use Python framework for building event-driven applications that interact with any MQ Broker
pykafka - Apache Kafka client for Python; high-level & low-level consumer/producer, with great performance.
fastkafka - FastKafka is a powerful and easy-to-use Python library for building asynchronous web services that interact with Kafka topics. Built on top of Pydantic, AIOKafka and AsyncAPI, FastKafka simplifies the process of writing producers and consumers for Kafka topics.
KQ - Kafka-based Job Queue for Python