faststream VS RabbitMQ

FastStream - https://github.com/airtai/faststream, a stream processing framework, already supports Kafka stream processing using the aiokafka library, as well as other brokers such as Redis, RabbitMQ, and NATS.

FastStream is a powerful and easy-to-use FOSS framework for building asynchronous services interacting with event streams such as Apache Kafka, RabbitMQ and NATS. It simplifies the process of writing producers and consumers for message queues, handling all the parsing, networking and documentation generation automatically.

Ready to join the FastStream revolution? Head over to our GitHub repository and show your support by starring it. By doing so, you'll stay in the loop with the latest developments, updates, and enhancements as we continue to refine and expand FastStream.

After two months of hard work, we presented the newly released FastStream framework at Infobip Shift conference and got featured at ShiftMag. The framework now supports both Apache Kafka and RabbitMQ, but also NATS protocol with the plan to add more protocols in the near future. The overall code is much cleaner and the implementation is streamlined with abstractions covering the common functionality across the protocols. We deprecated both FastKafka and Propan, but promised to fix bugs as long as needed. However, it seems like the community already decided to switch over to gain new functionalities.

faststream-gen(https://github.com/airtai/faststream-gen/) uses GPT models to automatically generate microservices using the FastStream(https://github.com/airtai/faststream) framework for Apache Kafka, RabbitMQ and NATS. Simply describe your microservice in plain English, and it will generate a production-level FastStream application ready to deploy in a few minutes and under $1 cost, together with unit and integration tests, documentation and Docker images.

faststream-gen is a Python library that uses generative AI to automatically generate FastStream applications. Simply describe your microservice in plain English, and faststream-gen will generate a production-level FastStream application ready to deploy in a few minutes and under $1 cost.

FastStream simplifies the process of writing producers and consumers for message queues, handling all the parsing, networking and documentation generation automatically. It is a new package based on the ideas and experiences gained from FastKafka and Propan. By joining our forces, we picked up the best from both packages and created a unified way to write services capable of processing streamed data regardless of the underlying protocol. We'll continue to maintain both packages, but new development will be in this project.

FastStream (https://github.com/airtai/faststream) is a new Python framework, born from Propan and FastKafka teams' collaboration (both are deprecated now). It extremely simplifies event-driven system development, handling all the parsing, networking, and documentation generation automatically. Now FastStream supports RabbitMQ and Kafka, but supported brokers are constantly growing (wait for NATS and Redis a bit). FastStream itself is a really great tool to build event-driven services. Also, it has a native FastAPI integration. Just create a StreamRouter (very close to APIRouter) and register event handlers the same with the regular HTTP-endpoints way:

Although they did not make it into production, I experimented with the RabbitMQ message broker, Python (Django, Flask), Kubernetes + minikube, JWT, and NGINX. This was a hobby project, but I intended to learn about microservices along the way.

Messaging/Queueing Systems (Amazon SQS, RabbitMQ, Beanstalkd)

Later, we discovered Propan, a library created by Nikita Pastukhov, which solved similar problems but for RabbitMQ. Recognizing the potential for collaboration, we joined forces with Nikita to build a unified library that could work seamlessly with both Kafka and RabbitMQ. And that's how FastStream came to be—a solution born out of the need for simplicity and efficiency in microservices development.

Inter-Service Communication: Middleware provides communication channels and protocols that enable microservices to communicate with each other. This can include message brokers like RabbitMQ, Apache Kafka, RPC frameworks like gRPC, or RESTful APIs.

This is an implementation of pub/sub. The publisher is on a separate project. The message broker is Azure Service Bus. We use NServiceBus for code implementation. I use rabbitMQ broker for local tests. Nothing I can do about the tech stack. This is more of a high level single project structure review 😅

RabbitMQ is a robust and highly configurable open-source message broker that implements the Advanced Message Queuing Protocol (AMQP).

When using third-party services, especially within a "transaction", it's often a good idea to use a persistent Message Queue (MQ) system like RabbitMQ. Go through all their tutorials to get a really good understanding of how message queues work and how they can be used to solve your problem.

The best technical solution to provide the event queues is to use a message-broker technology like RabbitMQ.