Testing a Kafka consumer with Avro schema messages in your Spring Boot application with Testcontainers

1. schema-registry

   1 7 2,436 10.0 Java

   Confluent Schema Registry for Kafka

   

   So that means we can configure the Kafka producer and consumer with an imaginary schema registry url, that only needs to start with “mock://” and you automatically get to work with the MockSchemaRegistryClient. This way you don't need to explicitly initiate the MockSchemaRegistryClient and configure everything accordingly. That also eradicates the need for the Confluent Schema Registry Container. Running the Kafka Testcontainer with the embedded Zookeeper, we no longer need an extra Zookeeper container and we are down to one Testcontainer for the messaging. This way I ended up with only two Testcontainers: Kafka and the database.

2. SaaSHub

   www.saashub.com featured

   SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

   

3. kafka-avro-without-registry

   2 1 25 1.5 Java

   Test Spring Kafka application (using Avro as a serialization mechanism) without the need for Confluent Schema Registry

   

   When you check on the internet for testing in the context of Spring Boot Kafka consumer and Avro schema, you find quite a few variants: using the MockSchemaRegistryClient, or writing your own custom Avro de-/serializers, or setting up a Testcontainers ecosystem with a Kafka, a Zookeeper and a Confluent Schema Registry, or using the EmbeddedKafka provided by Spring in the spring-kafka-test dependency.

4. kafka-consumer-testing

   3 1 8 0.0 Java

   demo application for testing a Kafka consumer with Avro message schema with Testcontainers using Kafka with embedded Zookeeper and no Schema Registry container

   

   Let me walk you through the smaller setup, simulating the situation above. We have a Spring Boot application, a PostgreSQL database and our Kafka consumer. The application needs to listen to Kafka messages for users that were added or modified, and has to update the database items accordingly. You can find the full code repository here. I added the UserEvent class for compilation purpse only, normally that would end up in your generated classes via the Avro plugin.

Suggest a related project