Load testing is hard, and the tools are not great. But why?

• har-to-k6

  2 117 7.0 JavaScript

  JSON config representation of K6 script

  

I agree that the popular load testing tools leave plenty to be desired, but have you given k6[1] a try? (Full disclosure: I'm one of the maintainers.)

Tests are written in JavaScript and there's support for HTTP, WebSockets and (unary) gRPC. You can easily script a combination of these protocols to mimic real world traffic.

Furthermore you can record a user flow with a browser extension[2] and convert the generated HAR file to a k6 script[3], which would give you an even closer real world scenario. The conversion is not perfect and depending on the service you might need to manually modify the script, but it would get you 90% of the way there.

[1]: https://github.com/loadimpact/k6

[2]: https://k6.io/docs/test-authoring/recording-a-session/browse...

[3]: https://github.com/loadimpact/har-to-k6

• Vegeta

  41 22,770 6.6 Go

  HTTP load testing tool and library. It's over 9000!

I've used vegeta (https://github.com/tsenart/vegeta) on a couple of projects. Easy to setup and use.

IMHO how you load test is almost as important as where you load test and understanding why. There are three separate reasons to do load testing:

1) Performance testing. Confirm the system does not degrade under a specified load and find out what performance can be expected under these circumstances. This is basically ensuring your system can handle X amount of traffic without issues and knowing your baseline performance. You should get the same kind of response times as you are getting from live server telemetry.

2) Stress testing. finding out what happens when the system is stressed beyond its specs. How does it degrade & where.

3) Reliability testing. Find out how your system breaks and when. The goal here is to try to break the system and test things like failover and making sure you don't lose or corrupt data. Better to die gracefully then abrubtly.

If that's new to you, you've been probably doing it wrong because those 3 require very different approaches.

I tend to avoid having to do load testing as it sucks up time without telling me much if interest. I instead opt for having decent telemetry on the live system. It will tell me how it performs and where the bottlenecks are. I can set alerts and take action when things degrade (e.g. because of a bad change). Besides there is no substitute for having real users doing real things with real data. And in any case, having telemetry is crucial to do any meaningful stress or reliability testing. Otherwise you just know it degrades without understanding why.

There are still valid reasons to do separate load testing of course but I seem to get away with mostly not doing this. When I do, vegeta is what I reach for first. I find most of the need for load testing is requirements related to SLAs or otherwise nervous product owners that need to be reassured. They tend to be more interested in performance tests (i.e. the happy case) whereas technically stress testing is where you learn stuff about your system.

• InfluxDB

  www.influxdata.com featured

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• shadowreader

  1 151 0.0 Python

  Serverless load testing for replaying website traffic. Powered by AWS Lambda.

  

At my workplace we're using a modified version of ShadowReader - https://github.com/edmunds/shadowreader; note it is AWS-specific. It works by re-running the load balancer logs, either in near-real time or a replay window. We use it test changes to our Solr search engine. It complements our Locust testing which is more about red-lining our servers and isn't as user-specific.

• tsung

  2 2,510 4.1 Erlang

  Tsung is a high-performance benchmark framework for various protocols including HTTP, XMPP, LDAP, etc.

  

Just thought I could add https://github.com/processone/tsung to the discussion.

Suggest a related project