Apache JMeter VS ZAP

You could use many methods to run performance tests on your app, one of the most popular being Apache JMeter. Let's use the benchmark numbers to compare Hanami and Rails.

Usually, I would let organic users be my load test. However, I am working on a project that has an anticipated load on a new-to-my-team stack, so I'm looking into ways to load test.

I've seen tools like k6 (https://k6.io/), Artillery (https://www.artillery.io), and JMeter (https://jmeter.apache.org/).

I've been using Artillery, but it's hard to visualize the results.

What do you use?

Apache JMeter: This tool is an open-source application built on Java, designed specifically to test load functionality and performance. Developed by the Apache Software Foundation, JMeter is versatile, able to simulate loads across a wide range of services and protocols such as HTTP, HTTPS, JDBC, LDAP, and SOAP. With an extensible core that can be tailored with plugins, it provides the flexibility needed for different testing scenarios. Its intuitive GUI makes it easy for testers to design test plans and visualize the results in various ways.

I'll use Apache JMeter to do this experiment:

Thanks for the tip. Hows that compare to this tool? https://jmeter.apache.org/

Apache JMeter: Download and install JMeter from the official website (https://jmeter.apache.org/). Java Development Kit (JDK): JMeter requires Java, so ensure you have the latest JDK installed on your system. Postman: Install Postman from the official website (https://www.postman.com/downloads/).

The test scenario consists of querying for descriptions of different offers. During the test, I will collect data on memory and GC parameters using jConsole. I will run the test scenario using jMeter, which additionally will allow me to measure response times.

FYI some of our people internally use Jmeter. https://jmeter.apache.org/

It's not flashy so it probably wont get the standard "we are going to milk you for data" plan

Issue #5682Pull request #717 - Open Model Thread Group: avoid skipping rows from CSV Data Set Config

Performance and load testing are essential parts of GraphQL API testing. It ensures APIs can handle expected traffic volumes and respond within acceptable timeframes. You can use tools like Apache JMeter or Gatling to generate realistic loads and evaluate the API's performance under different scenarios. Techniques like batched queries and caching can help mitigate this issue.

I use ZAP [1] with the OAST add-on for this at the moment. I admit the UX isn't perfect, but it serves my purpose.

If I also want control over the responses (e.g. return a 401 status code for every fifth request), I have a custom extender script [2] for that.

[1]: https://www.zaproxy.org/

Implement tools like Burp Suite or OWASP ZAP for in-depth security scanning of your APIs.

OWASP ZAP

The use of capital punctuation implies a warning? an alert? Would this same response be warranted for Burp which is also a commercial, closed source product?

If this is an issue for some, then ZAP being open source[1] maybe favourable.

That said, Burp is the defacto tool for a reason - it's best in class. Every pentester I know, including myself, has a paid subscription. The fact that it's closed source hasn't been an issue.

[1] https://github.com/zaproxy/zaproxy

Briefly reviewed your product. Seems like OWASP ZAP is your competition: https://www.zaproxy.org/

It runs entirely in the browser so it uses the browser "native" frameworks.

Dynamic analysis involves testing your application while it's running. Tools like OWASP ZAP and Burp Suite can help identify vulnerabilities like SQL injection or Cross-Site Scripting by sending malicious requests to your application and analyzing the responses.

> Lying is not an embellishment or puffery, it's a lie. Engaging a company for a 3 day pen test that's totally insufficient, that would be an embellishment.

I agree, but if the RFP question was phrased "have you done penetration testing?" then that leaves a lot of room for embellishment. If the question is "do you have SOC2 certification?" and you answer "yes" untruthfully, then that is a lie. If they ask for the SOC2 or pentest report and you give them a falsified document, that's where you're (probably) committing fraud.

> One of the most important part of pen tests is that they are external.

AWS/Google/etc have internal security teams doing their pen tests, so no, this isn't true.

> Just doing your job as an engineer and looking for bugs is not a pen test.

What about an engineer spending an afternoon running ZAP[0]?

> It's like saying, "what is an audit really? We have accountants and they check our books for anomalies."

Yeah, which is why you don't just ask a company "do you keep track of your finances?" if you're investing in them, you request external auditors.

[0] https://www.zaproxy.org/

In addition to manual security reviews, you can also implement DevSecOps practices to automate security checks. For example, you can set up a CI/CD pipeline to run static code analysis tools like CodeQL and automatically run penetration tests using tools like OWASP ZAP.

OWASP ZAP (open source)

I would start by installing Burp Suite or OWASP Zap and seeing what the actual messages look like