confluent-kafka-python
quix-streams
Our great sponsors
confluent-kafka-python | quix-streams | |
---|---|---|
9 | 25 | |
3,632 | 570 | |
1.8% | 35.3% | |
6.3 | 9.0 | |
2 days ago | 2 days ago | |
Python | Python | |
GNU General Public License v3.0 or later | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
confluent-kafka-python
-
Show HN: Confluent Kafka support added to FastStream v0.4.0rc0
Responding to popular demand, the latest 0.4.0rc0 version introduces support for Kafka stream processing using Confluent Kafka's Python library - https://github.com/confluentinc/confluent-kafka-python.
-
confluent-kafka-python VS quix-streams - a user suggested alternative
2 projects | 7 Dec 2023
-
Recently joined a DE team and I've been asked to study async, multiprocessing, queuing, and Kafka. Can anybody tell me how to proceed and also share resources that I can use.
- confluent kafka library (https://github.com/confluentinc/confluent-kafka-python) uses librdkafka, and is faster that kafka-python - https://www.bytewax.io/(https://github.com/bytewax/bytewax) is another option. Handles the dirty work of exchanging data across processes for parallel stateful processing
-
New to kafka..
Here are examples https://github.com/confluentinc/confluent-kafka-python
-
On Efficiently Partitioning a Topic in Apache Kafka
I just wanted to mention that pykafka is currently unmaintained and archived on GitHub:
https://github.com/Parsely/pykafka
pykafka was originally developed and maintained by my team at Parse.ly, but we no longer maintain it. We instead encourage folks to use confluent-kafka-python, which is what we have ourselves switched to in our production systems:
https://github.com/confluentinc/confluent-kafka-python
(pykafka was developed at a time before Confluent invested in their own Python binding. Some of the history of the project is described in this 2016 blog post[1] and our original 2015 announcement[2].)
[1]: https://blog.parse.ly/pykafka-now/
[2]: https://blog.parse.ly/announcing-pykafka-python-support-for-...
-
Is there a Python API for event-driven Kafka consumer?
confluent-kafka
-
Kafla producer over internet
I also found that there is a library provided by confluent https://github.com/confluentinc/confluent-kafka-python which maybe helpful here. Is that right? I see a section "SSL certificates" but couldnt get much insight out of this.
-
Confluent Kafka Python Schema Registry: Why the consumer does not need it?
ProtobugDeserializer does not allow anything but the protoBuf message type (see here):
-
Learning Apache Kafka
I'd start off with some simple producer/consumer code to get an understanding of interacting with Kafka's APIs from Python - there's a Python client with some good examples of the producer and consumer as well as more complex examples.
quix-streams
- Show HN: Streaming DataFrames–a Pandas-like syntax for real-time data
- FLaNK AI-April 22, 2024
-
Airflow VS quix-streams - a user suggested alternative
2 projects | 7 Dec 2023
Airflow for Streaming
-
Apache Pulsar VS quix-streams - a user suggested alternative
2 projects | 7 Dec 2023
-
redpanda VS quix-streams - a user suggested alternative
2 projects | 7 Dec 2023
-
ApacheKafka VS quix-streams - a user suggested alternative
2 projects | 7 Dec 2023
-
flink-statefun VS quix-streams - a user suggested alternative
2 projects | 7 Dec 2023
-
Apache Spark VS quix-streams - a user suggested alternative
2 projects | 7 Dec 2023
-
beam VS quix-streams - a user suggested alternative
2 projects | 7 Dec 2023
-
debezium VS quix-streams - a user suggested alternative
2 projects | 7 Dec 2023
Debezium is a change data capture framework for streaming. Connect it to your databases to detect changes and produce those change events to Kafka. Quix Streams is a Python stream processing library for ML and AI applications. It builds on the Confluent Python library to add a state store with RocksDB and adds a Streaming DataFrames API with declarative operations like Windows. It is designed for analytics and data engineering workloads. Use it together with Debezium to process your CDC data in real-time.
What are some alternatives?
Confluent Kafka Golang Client - Confluent's Apache Kafka Golang client
quix-samples - Library samples repository of Quix. Explore and Deploy them easily on https://portal.platform.quix.ai
aiokafka - asyncio client for kafka
kafka-python - Python client for Apache Kafka
csv-import - The open-source CSV importer, maintained by @tableflowhq
demo-scene - 👾Scripts and samples to support Confluent Demos and Talks. ⚠️Might be rough around the edges ;-) 👉For automated tutorials and QA'd code, see https://github.com/confluentinc/examples/
qr-code - A no-framework, no-dependencies, customizable, animate-able, SVG-based <qr-code> HTML element.
pykafka - Apache Kafka client for Python; high-level & low-level consumer/producer, with great performance.
ml-runtimes
KQ - Kafka-based Job Queue for Python
bytewax - Python Stream Processing