artillery VS Newman

Usually, I would let organic users be my load test. However, I am working on a project that has an anticipated load on a new-to-my-team stack, so I'm looking into ways to load test.

I've seen tools like k6 (https://k6.io/), Artillery (https://www.artillery.io), and JMeter (https://jmeter.apache.org/).

I've been using Artillery, but it's hard to visualize the results.

What do you use?

This week was Tracetest’s first-ever Launch Week. We’ve been working on a major integration with Artillery for the last month and our team is beyond excited to share it with you all!

I found a tool for load testing called Artillery. Following this guide I installed Artillery and began research for the test configuration.

This is a great blog post! just taking the opportunity here to comment on this:

> Finally for full scale high fidelity load tests there are relatively few tools out there for browser based load testing.

It exists as of a few months ago and it's fully open source: https://github.com/artilleryio/artillery (I'm the lead dev). You write a Playwright script, then run it in your own AWS account on serverless Fargate and scale it out horizontally as you see fit. Artillery takes care of spinning up and down all of the infra. It will also automatically grab and report Core Web Vitals for you from all those browser sessions, and we just released support for tracing so you can dig into the details of each session if you want to (OpenTelemetry based so works with most vendors- Datadago APM, New Relic etc)

So not to stress test Momento or AWS' Lambda, I wanted to build a small but stable 10-minute workload that hits the Momento Topic API and then let Momento trigger the FunctionURL to run the Lambda code. I wrote a small Artillery config file that ramps up to 20 users and then sustains that for the duration. Again, the script is simple to trigger the work.

Tool Spotlight: Featuring insights on how Artillery and Gitpod can enhance and streamline the benchmarking process.

curl is fantastic. There's also HTTPStat which provides a waterfall visualization on top of curl timings: https://github.com/reorx/httpstat

There's also Skytrace (made by yours truly), which provides timing info as a waterfall visualization inspired by HTTPStat + lots more (syntax highlighting for responses, built-in JMESPath support, command-line assertions and checks etc) - https://github.com/artilleryio/artillery/tree/main/packages/...

https://www.artillery.io/

I know it's an annoying answer, but it depends on your application. The only true way to know is to test it using a load tester like artillery. Measuring performance is a fundamental part of any optimisation (otherwise how do you know?), so it's a great idea to be using tools like this anyway.

For this test I have used artillery.io with the following configuration:

There are several testing approaches you can take to ascertain the functionality of the auth API. One way would be to automate the testing using unit tests with tools like Jest and Supertest. Alternatively, you can make use of API clients like Postman or the Thunder Client VS Code extension to test the API.

I'm excited to be speaking at POSTCON, where I will dive into the world of real-time data using Zoom's innovative WebSockets. We'll seamlessly test these functionalities with the help of POSTMAN. This session is designed to give you a glimpse into how we at Zoom are enhancing the real-time capabilities of our platform to ensure faster, more reliable communication. During this session, we'll explore the limitations with Webhooks and why WebSockets are becoming a preferable alternative for real-time, bi-directional communication.

The results of the experiment below were based on reproducing more than 100 cold and approximately 100.000 warm starts with Lambda function with 1024 MB memory setting for the duration of 1 hour. For it I used the load test tool hey, but you can use whatever tool you want, like Serverless-artillery or Postman.

Testing Tools: Select suitable automated testing tools, such as OWASP Zap and Postman, for security and functionality testing.

Postman - API platform for easy endpoint testing

The results of the experiment to retrieve the existing product from the database by its id see GetProductByIdViaAuroraServerlessV2DataApiHandler with Lambda function with 1024 MB memory setting were based on reproducing more than 100 cold and approximately 10.000 warm starts with experiment which ran for approximately 1 hour. For it (and experiments from my previous article) I used the load test tool hey, but you can use whatever tool you want, like Serverless-artillery or Postman. We won't enable SnapStart on the Lambda function first.

Use tools like Postman or Insomnia to test the API endpoints and ensure they behave as expected.

Products are available on our Strapi server. We need to be able to send HTTP requests that will allow clients or users to perform CRUD operations on these product resources. Postman will be our tool for making requests to the Strapi REST API.

We will be performing all of the authentication requests manually, however for testing purposes, you might want to use an API testing tool such as Postman or Insomnia.

As you can see in the above diagram, a lambda function is scheduled to be executed periodically by EventBridge. This lambda function retrieves the assets exported from Postman (test collection, environment, and global variables), injects secrets from the secrets manager and executes the tests using the newman npm package, and, in case of failures, updates metrics in CloudWatch and stores test results in the S3 bucket. An alarm is triggered in case the metrics exceed a threshold (in this case, a count of 1).