artillery VS Playwright

Usually, I would let organic users be my load test. However, I am working on a project that has an anticipated load on a new-to-my-team stack, so I'm looking into ways to load test.

I've seen tools like k6 (https://k6.io/), Artillery (https://www.artillery.io), and JMeter (https://jmeter.apache.org/).

I've been using Artillery, but it's hard to visualize the results.

What do you use?

This week was Tracetest’s first-ever Launch Week. We’ve been working on a major integration with Artillery for the last month and our team is beyond excited to share it with you all!

I found a tool for load testing called Artillery. Following this guide I installed Artillery and began research for the test configuration.

This is a great blog post! just taking the opportunity here to comment on this:

> Finally for full scale high fidelity load tests there are relatively few tools out there for browser based load testing.

It exists as of a few months ago and it's fully open source: https://github.com/artilleryio/artillery (I'm the lead dev). You write a Playwright script, then run it in your own AWS account on serverless Fargate and scale it out horizontally as you see fit. Artillery takes care of spinning up and down all of the infra. It will also automatically grab and report Core Web Vitals for you from all those browser sessions, and we just released support for tracing so you can dig into the details of each session if you want to (OpenTelemetry based so works with most vendors- Datadago APM, New Relic etc)

So not to stress test Momento or AWS' Lambda, I wanted to build a small but stable 10-minute workload that hits the Momento Topic API and then let Momento trigger the FunctionURL to run the Lambda code. I wrote a small Artillery config file that ramps up to 20 users and then sustains that for the duration. Again, the script is simple to trigger the work.

Tool Spotlight: Featuring insights on how Artillery and Gitpod can enhance and streamline the benchmarking process.

curl is fantastic. There's also HTTPStat which provides a waterfall visualization on top of curl timings: https://github.com/reorx/httpstat

There's also Skytrace (made by yours truly), which provides timing info as a waterfall visualization inspired by HTTPStat + lots more (syntax highlighting for responses, built-in JMESPath support, command-line assertions and checks etc) - https://github.com/artilleryio/artillery/tree/main/packages/...

https://www.artillery.io/

I know it's an annoying answer, but it depends on your application. The only true way to know is to test it using a load tester like artillery. Measuring performance is a fundamental part of any optimisation (otherwise how do you know?), so it's a great idea to be using tools like this anyway.

For this test I have used artillery.io with the following configuration:

We start with a project that was bootstrapped with npx create-next-app. For the E2E test we use Playwright and set it up as described in the testing guide provided by Next.js.

Playwright is a powerful tool developed by Microsoft, it allows developers to write reliable end-to-end tests and perform browser automation tasks with ease. What sets Playwright apart is its ability to work seamlessly across multiple browsers (Chrome, Firefox, and WebKit), it provides a consistent and efficient way to interact with web pages, extract data, and automate repetitive tasks. Moreover, it supports various programming languages such as Node.js, Python, Java, and .NET, that’s making it a versatile choice for web scraping projects. Whether you're scraping public data for analysis, building a web crawler, or automating manual workflows, Playwright has you covered.

The consensus I could gather is either use playwright or use a workaround to solve it in the puppeteer layer. The root cause of the bug is a websocket size limitation on the CDP protocol for chromium.

With the advent of tools like Puppeteer and now Playwright, end-to-end testing has become much easier and more reliable. For anyone who's used Selenium in the past, you know what I'm talking about. Puppeteer has opened the way in terms of E2E tooling, but Playwright has taken it to the next level and made it easier to await for certain selectors or conditions to be fulfilled (via locators), thus making tests more reliable and less flaky. Also, it's a game changer that it introduced a test-runner - this made the integration between the headless browser and the actual test code much smoother.

In this tutorial, our main focus will be on Playwright web scraping. So what is Playwright? It’s a handy framework created by Microsoft. It's known for making web interactions more streamlined and works reliably with all the latest browsers like WebKit, Chromium, and Firefox. You can also run tests in headless or headed mode and emulate native mobile environments like Google Chrome for Android and Mobile Safari.

// playwright.config.ts import { defineConfig } from "@playwright/test"; /** * See https://playwright.dev/docs/test-configuration. */ export default defineConfig({ testDir: "./src/pages", reporter: "list", use: { baseURL: "http://localhost:5432/", }, timeout: process.env.CI ? 10000 : 4000, // ... more options });

Issues with Playwright

Fast and reliable end-to-end testing for modern web apps | Playwright

This means you cannot place test files outside of this directory, which was brought up as a question on Github some time ago. Initially, I thought it would be nice to add another folder in the repo called "scripts", but Playwright does not allow multiple testDir values.

4.Playwright: Playwright is a browser automation library by Microsoft. Key Features: Supports Chromium, Firefox, and WebKit. Provides cross-browser testing capabilities. Allows automating web, mobile, and desktop applications