pact-js VS dredd

First, he shows the official pact.io websites. Then, he clicks on the “View on Github” button by selecting Node JS and Javascript from the list of options.

It just just enough abstraction over the basics of converting your HTTP calls to GraphQL queries and mutations, but ALL of the parsing of responses is on you. I’m well versed in parsing JSON in Elm. I’m also familiar with the compiler errors as well as runtime errors you get with JSON that doesn’t match up to what you designed. At some point I’ll probably have to move beyond the unit tests and add contact tests, maybe via Pact.js.

> I also wasn't aware that "unit" referred to an isolated test, not to the SUT.

I'm with you. That claim is unsubstantiated. It seems to trace to the belief that the first unit tests were XUnit family, thus were SUnit for Scheme. But Kent Beck made it pretty clear that SUnit "units" were classes.

https://web.archive.org/web/20150315073817/http://www.xprogr...

There were unit tests before that. SUnit took its name from common parlance, not vice versa. It was a strange naming convention, given that the unit testing framework could be used to test anything and not just units. Much like the slightly older Test Anything Protocol (TAP) could.

> [on unit tests] This does lead to a lot of work maintaining them whenever the implementation changes, but this is a necessary chore because of the value they provide.

I disagree. Unit tests can still be behavioral. Then they change whenever the behavior changes. They should still work with a mere implementation change.

> This is why I still think that the traditional test pyramid is the best model to follow.

I'll disagree a little with that, too. I think a newer test pyramid that uses contract testing to verify integrations is better. The notion of contract tests is much newer than the pyramids and, properly applied, can speed up feedback by orders of magnitude while also cutting debugging time and maintenance by orders of magnitude.

On that front, I love what Pact is doing and would like to see more competition in the area. Hottest thing in testing since Cypress/Playwright . . .

https://pact.io

I've worked in places where Pact [0] was used for testing services developed by different teams (external) and teams themselves (internal)

[0] https://pact.io/

I'd never heard anyone call those "integrity tests" before. I think "contract test" is more common.

Assuming I understood you, that is.

I've been telling everyone to look at Pact to make contract testing easier to organize and maintain and to make it easier to trigger in the other tests in CI when an interface's behavior changes. They haven't offered me a commission yet. ;-)

https://pact.io

We use pact and since introducing it we have significantly increased velocity and reduced test cycles as it catches things very early. For system tests we hand write them using whatever test frameworks the team is used to.

There is also Pact https://pact.io/ for a language agnostic pact testing.

When you wish to focus on the contract ( which kind of field is required, ...), you shoud use contract testing frameworks. As you seem to leverage a microservices, a consumer driven contract testing approach with a framework like Pact.js is recommended.

Dredd: used to test APIs based on the API blueprint or OpenAPI specification, to ensure implementation matches the specification.

Here are three tools that you can use to generate example API requests and responses from OpenAPI specifications. These tools should work well even if your schemas are deeply nested: Nswag (Command Line and GUI): Nswag is a Swagger/OpenAPI toolchain for .NET, TypeScript, and other platforms. It supports code generation, client generation, and API documentation. You can use NswagStudio, which is a graphical interface, or you can use the command line tool called "NSwag.exe" for generating example API requests and responses. GitHub: https://github.com/RicoSuter/NJsonSchema NswagStudio: https://github.com/RicoSuter/NSwag/wiki/NSwagStudio Dredd (Command Line): Dredd is a language-agnostic command-line tool for validating API descriptions against backend implementations. It supports OpenAPI, Swagger, and API Blueprint formats. Dredd can generate example requests and responses and validate whether your API implementation conforms to the API description. GitHub: https://github.com/apiaryio/dredd Documentation: https://dredd.org/en/latest/ Stoplight Studio (GUI): Stoplight Studio is a modern API design and documentation platform that supports OpenAPI and JSON Schema. It allows you to create, edit, and validate OpenAPI specifications and provides a powerful visual interface for generating example API requests and responses. Website: https://stoplight.io/studio/ GitHub: https://github.com/stoplightio/studio These tools should provide you with the ability to generate example API requests and responses from your OpenAPI specifications and handle deeply nested schemas.

If you want to make sure the server implements a certain contract like there's an handler responding to a GET request to /API/what/ever I'd rather use something else. To be completely honest this is a topic I'm currently also searching for a really good solution but what I found so far (and looks promising) is https://dredd.org/ or https://microcks.io/ Both support OpenAPI testing so you can specify the contract as an OpenAPI spec and validate your server against it.

Consolidating the API specification with OpenAPI was a turning point for the project. From that moment we were able to run mock servers to build and test the UI before integrating with the backend, and we were able to validate the backend implementation against the specification. We used prism to run mock servers, and Dredd to validate the server implementation (these days I’d rather use schemathesis).

In this approach, you produce an API specification first, then you build the API against the specification, and then you validate your implementation against the specification using automated API testing tools. This is the most reliable approach for building API servers, since it’s the only one that holds the server accountable and validates the implementation against the source of truth. Unfortunately, this approach isn’t as common as it should be. One of the reasons why it isn’t so common is because it requires you to produce the API specification first, which, as we saw earlier, puts off many developers who don’t know how to work with OpenAPI. However, like I said before, generating OpenAPI specifications doesn’t need to be painful since you can use tools for that. In this approach, you use automated API testing tools to validate your implementation. Tools like Dredd and schemathesis. These tools work by parsing your API specification and automatically generating tests that ensure your implementation complies with the specification. They look at every aspect of your API implementation, including use of headers, status codes, compliance with schemas, and so on. The most advanced of these tools at the moment is schemathesis, which I highly encourage you to check out.

It's missing the greatest API testing classic Dredd! Other than that the best API testing tool I've used so far is schemathesis. It works by looking at your API specification and automatically launching hundreds of tests per endpoint. It also leverages advanced OpenAPI documentation strategies such as links to test the relationship between various endpoints.

One more tip for the backend developers: make sure the API implementation is tested against the API specification using contract-testing tools such as Dredd or Schemathesis. I specially recommend schemathesis as it's a lot more comprehensive. I recommend you run those tests in the CI and require them to pass before they can merge their API changes. This is the only reliable way to ensure the API works as expected.

The other thing you want to make sure is that the server is implementing the API correctly. In this space, you can use tools such as Dredd and schemathesis, which look at the API specification and automatically test the server implementation against it.

Schemaless schemas make testing difficult. Tools like Dredd and Schemathesis rely on your API documentation to generate tests and validate your API responses. A collection of free-form arrays like the above model will pass nearly every test, even if the length of the arrays or their contents are wrong. Schemaless schemas are also useless for API mocking, which is a fundamental part of building reliable API integrations.

Dredd: this is the classic API testing tool and it's been around for years. Dredd works by looking at your API specification and figuring out what tests need to be generated to validate your API implementation. You don't need to write any additional code, although you may want to create your own custom hooks to customise Dredd's behaviour. Dredd hooks are useful for example to test resource endpoints (the likes of /todo/{todo_id}) and to clean up your database from any resources created during the test suite. I wrote a tutorial on how to write Dredd hooks which you may find useful.