Recently joined a DE team and I've been asked to study async, multiprocessing, queuing, and Kafka. Can anybody tell me how to proceed and also share resources that I can use.

• quix-streams

  25 530 9.0 Python

  A Python library for building containerized ML and Generative AI applications with Apache Kafka.

  

- Quix streams

• aiokafka

  3 1,043 8.6 Python

  asyncio client for kafka

  

It’s hard to understand exactly what is meant by what you posted but could they be referring to asyncio libraries for use with Kafka in this process? For example, https://github.com/aio-libs/aiokafka

• InfluxDB

  www.influxdata.com sponsored

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• confluent-kafka-python

  9 3,628 6.9 Python

  Confluent's Kafka Python Client

  

- confluent kafka library (https://github.com/confluentinc/confluent-kafka-python) uses librdkafka, and is faster that kafka-python - https://www.bytewax.io/(https://github.com/bytewax/bytewax) is another option. Handles the dirty work of exchanging data across processes for parallel stateful processing

Suggest a related project