xk6-kafka
xk6
Our great sponsors
xk6-kafka | xk6 | |
---|---|---|
4 | 2 | |
125 | 174 | |
- | 3.4% | |
6.3 | 5.9 | |
15 days ago | about 1 month ago | |
Go | Go | |
Apache License 2.0 | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
xk6-kafka
-
Performance testing Kafka server using xk6-kafka
import { Writer, Reader, SCHEMA_TYPE_STRING, SchemaRegistry, GROUP_BALANCER_ROUND_ROBIN, BALANCER_ROUND_ROBIN, } from "k6/x/kafka"; import { check, sleep } from "k6"; const bootstrapServers = [ 'kafka-bootsrap-1:9001', ]; let vus_amount = 3; let total_written = 0; let total_read = 0; export const options = { vus: vus_amount, iterations: "3", thresholds: { kafka_writer_error_count: ["count == 0"], kafka_reader_error_count: ["count == 0"], }, }; const topicName = "my-topic"; const schemaRegistry = new SchemaRegistry(); const producer = new Writer({ brokers: bootstrapServers, topic: topicName, balancer: BALANCER_ROUND_ROBIN, // or pick another balancer https://github.com/mostafa/xk6-kafka/blob/main/api-docs/index.d.ts#L66 // ... auth config }); export function teardown(data) { producer.close(); } export default function () { const consumer = new Reader({ brokers: bootstrapServers, // it is important to set groupID, groupTopics and groupBalancers when using Kafka bootstrap server // topic ReaderConfig param doesn't quite work with bootstrap server groupID: topicName + "-group", groupTopics: [topicName], groupBalancers: [GROUP_BALANCER_ROUND_ROBIN], // or pick different balancer https://github.com/mostafa/xk6-kafka/blob/main/api-docs/index.d.ts#L75 }); let messageAmount = 1000; if (__VU == 1) { console.log('VU 1, writing messages. Iter ' + __ITER); for (let index = 0; index < messageAmount; index++) { let messages = [ { value: schemaRegistry.serialize({ data: "test-value-string-" + index + "-vu-" + __VU + "-iter-" + __ITER, schemaType: SCHEMA_TYPE_STRING, }), }, ]; producer.produce({ messages: messages }); total_written += messages.length; } } let consumerMsgAmount = Math.floor(messageAmount / vus_amount); let messages = consumer.consume({ limit: consumerMsgAmount}); total_read += messages.length; console.log("Amount of msgs received: " + messages.length + ", VU " + __VU + ", iter " + __ITER); check(messages, { "all messages returned": (msgs) => msgs.length == consumerMsgAmount, }); let firstMessageValue = schemaRegistry.deserialize({ data: messages[0].value, schemaType: SCHEMA_TYPE_STRING, }); let lastMessageValue = schemaRegistry.deserialize({ data: messages[consumerMsgAmount - 1].value, schemaType: SCHEMA_TYPE_STRING, }); check(messages[0], { "Topic equals to": (msg) => msg["topic"] == topicName }); console.log("First msg value " + firstMessageValue + ", offset" + messages[0]["offset"] + ", partition " + messages[0]["partition"] + ", VU " + __VU + ", iter " + __ITER); console.log("Last msg value " + lastMessageValue + ", offset" + messages[consumerMsgAmount - 1]["offset"] + ", partition " + messages[0]["partition"] + ", VU " + __VU + ", iter " + __ITER); consumer.close(); }
-
🔥 What is Apache Kafka and how to perform performance tests on it (Part 2) 👨🏻💻
Apache Kafka, like any other service, platform or server, has its performance limitations. In order to know them, you need to run performance tests on Apache Kafka's Cluster, which will show acceptation thresholds. One project that can help with this is xk6-kafka.
-
xk6-kafka v0.12.0 is released! 🎉
I am thrilled to announce that I released xk6-kafka v0.12.0, a complete revamp of the JavaScript API plus many other features and fixes. You can read more about it on the changelog of the release: https://github.com/mostafa/xk6-kafka/releases/tag/v0.12.0
-
How to Load Test Your Kafka Producers and Consumers using k6
The xk6-kafka extension provides some convenient functions for interacting with Kafka Producers and Consumers. It serves as a producer that can send a high volume of messages per second, allowing you to monitor the system under test (SUT) and test how the application will keep up with the load.
xk6
-
How to Load Test Your Kafka Producers and Consumers using k6
Recently, k6 started supporting k6 extensions to extend k6 capabilities for other cases required by the community. The community has already built plenty of extensions. k6 extensions are written in Go, and many of them are reusing existing Go libraries.
-
Benchmarking Redis with k6
Previously, I have covered an article on Load Testing SQL Databases with k6. For your information, from k6 version 0.29.0 onwards, you can write a k6 Go extension and build your own k6 binaries. This comes in handy as you can use a single framework for load testing different protocols, such as ZMTQ, SQL, Avro, MLLP, etc.
What are some alternatives?
k6-reporter - Output K6 test run results as formatted & easy to read HTML reports
xk6-tarantool - ⚙️ A k6 extension for Tarantool
ddosify - Effortless Kubernetes Monitoring and Performance Testing. Available on CLI, Self-Hosted, and Cloud
xk6-tcp - A k6 extension for sending data to TCP port
postman-to-k6 - Converts Postman collections to k6 script code
Redis - Redis is an in-memory database that persists on disk. The data model is key-value, but many different kind of values are supported: Strings, Lists, Sets, Sorted Sets, Hashes, Streams, HyperLogLogs, Bitmaps.
xk6-redis - A k6 extension to test the performance of a Redis instance.
kafkarest-processor - Apache NiFi 1.10.0 Processor to consume 1 Kafka message at a time, easily to tie into a REST Proxy
xk6-cable - A k6 extension for testing Action Cable and AnyCable functionality
singlestore-logistics-sim - Scalable package delivery logistics simulator built using SingleStore and Vectorized Redpanda