xk6-kafka
k6-reporter
Our great sponsors
xk6-kafka | k6-reporter | |
---|---|---|
4 | 1 | |
125 | 350 | |
- | - | |
6.3 | 0.0 | |
13 days ago | about 1 month ago | |
Go | EJS | |
Apache License 2.0 | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
xk6-kafka
-
Performance testing Kafka server using xk6-kafka
import { Writer, Reader, SCHEMA_TYPE_STRING, SchemaRegistry, GROUP_BALANCER_ROUND_ROBIN, BALANCER_ROUND_ROBIN, } from "k6/x/kafka"; import { check, sleep } from "k6"; const bootstrapServers = [ 'kafka-bootsrap-1:9001', ]; let vus_amount = 3; let total_written = 0; let total_read = 0; export const options = { vus: vus_amount, iterations: "3", thresholds: { kafka_writer_error_count: ["count == 0"], kafka_reader_error_count: ["count == 0"], }, }; const topicName = "my-topic"; const schemaRegistry = new SchemaRegistry(); const producer = new Writer({ brokers: bootstrapServers, topic: topicName, balancer: BALANCER_ROUND_ROBIN, // or pick another balancer https://github.com/mostafa/xk6-kafka/blob/main/api-docs/index.d.ts#L66 // ... auth config }); export function teardown(data) { producer.close(); } export default function () { const consumer = new Reader({ brokers: bootstrapServers, // it is important to set groupID, groupTopics and groupBalancers when using Kafka bootstrap server // topic ReaderConfig param doesn't quite work with bootstrap server groupID: topicName + "-group", groupTopics: [topicName], groupBalancers: [GROUP_BALANCER_ROUND_ROBIN], // or pick different balancer https://github.com/mostafa/xk6-kafka/blob/main/api-docs/index.d.ts#L75 }); let messageAmount = 1000; if (__VU == 1) { console.log('VU 1, writing messages. Iter ' + __ITER); for (let index = 0; index < messageAmount; index++) { let messages = [ { value: schemaRegistry.serialize({ data: "test-value-string-" + index + "-vu-" + __VU + "-iter-" + __ITER, schemaType: SCHEMA_TYPE_STRING, }), }, ]; producer.produce({ messages: messages }); total_written += messages.length; } } let consumerMsgAmount = Math.floor(messageAmount / vus_amount); let messages = consumer.consume({ limit: consumerMsgAmount}); total_read += messages.length; console.log("Amount of msgs received: " + messages.length + ", VU " + __VU + ", iter " + __ITER); check(messages, { "all messages returned": (msgs) => msgs.length == consumerMsgAmount, }); let firstMessageValue = schemaRegistry.deserialize({ data: messages[0].value, schemaType: SCHEMA_TYPE_STRING, }); let lastMessageValue = schemaRegistry.deserialize({ data: messages[consumerMsgAmount - 1].value, schemaType: SCHEMA_TYPE_STRING, }); check(messages[0], { "Topic equals to": (msg) => msg["topic"] == topicName }); console.log("First msg value " + firstMessageValue + ", offset" + messages[0]["offset"] + ", partition " + messages[0]["partition"] + ", VU " + __VU + ", iter " + __ITER); console.log("Last msg value " + lastMessageValue + ", offset" + messages[consumerMsgAmount - 1]["offset"] + ", partition " + messages[0]["partition"] + ", VU " + __VU + ", iter " + __ITER); consumer.close(); }
-
🔥 What is Apache Kafka and how to perform performance tests on it (Part 2) 👨🏻💻
Apache Kafka, like any other service, platform or server, has its performance limitations. In order to know them, you need to run performance tests on Apache Kafka's Cluster, which will show acceptation thresholds. One project that can help with this is xk6-kafka.
-
xk6-kafka v0.12.0 is released! 🎉
I am thrilled to announce that I released xk6-kafka v0.12.0, a complete revamp of the JavaScript API plus many other features and fixes. You can read more about it on the changelog of the release: https://github.com/mostafa/xk6-kafka/releases/tag/v0.12.0
-
How to Load Test Your Kafka Producers and Consumers using k6
The xk6-kafka extension provides some convenient functions for interacting with Kafka Producers and Consumers. It serves as a producer that can send a high volume of messages per second, allowing you to monitor the system under test (SUT) and test how the application will keep up with the load.
k6-reporter
-
QA - no excuses!
HTML Results are obtained converting k6 output to HTML. This is done using the k6-reporter project. The HTML generated is not compliant with Jenkins Content Security Policy so I modified it for allowing correct visualization. If you do not need this feature, feel free to remove hudson.model.DirectoryBrowserSupport.CSP directive inside /jenkins/Dockerfile file.
What are some alternatives?
ddosify - Effortless Kubernetes Monitoring and Performance Testing. Available on CLI, Self-Hosted, and Cloud
k6-examples - Project using K6 and Javascript to create scenarios of Load and Stress Test
postman-to-k6 - Converts Postman collections to k6 script code
jenkins-docker-qa - A zero to hero guide for installing and running Jenkins inside docker on Windows
xk6 - Build k6 with extensions
pitstop - This repo contains a sample application based on a Garage Management System for Pitstop - a fictitious garage. The primary goal of this sample is to demonstrate several software-architecture concepts like: Microservices, CQRS, Event Sourcing, Domain Driven Design (DDD), Eventual Consistency.
xk6-redis - A k6 extension to test the performance of a Redis instance.
kafkarest-processor - Apache NiFi 1.10.0 Processor to consume 1 Kafka message at a time, easily to tie into a REST Proxy
hub-feedback - Feedback and bug reports for the Docker Hub
singlestore-logistics-sim - Scalable package delivery logistics simulator built using SingleStore and Vectorized Redpanda
Grafana - The open and composable observability and data visualization platform. Visualize metrics, logs, and traces from multiple sources like Prometheus, Loki, Elasticsearch, InfluxDB, Postgres and many more.