Our great sponsors
-
-
-
WorkOS
The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
-
-
-
-
-
venom
🐍 Manage and run your integration tests with efficiency - Venom run executors (script, HTTP Request, web, imap, etc... ) and assertions (by ovh)
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
Oh, nice to meet you!
Our Matcher functionality was actually inspired by Hurl (you call them predicates!)! We would take your inspiration from us as a huge compliment :)
The main difference between Hurl and Step CI is from our persepective an ideological one. What do you think?
Hurl is a CLI tool, while Step CI is a library. This means that you can include our Test Runner (https://github.com/stepci/runner) and use it with other tools. For example, you can combine it with testing frameworks like Mocha, Jest, and Ava (https://github.com/stepci/stepci/blob/main/docs/using-test-t...) or build your own platform on top of it.
That's what's so nice about open source.
We’re sorry to hear that! Would you mind sharing some context with us?
Our OpenAPI (Swagger) integration is currently in Beta and only available through our website. We acknowledge, that it is not ready for prime-time just yet
Everything we do is open-source. You can see the list of issues in the repository and contribute if possible: https://github.com/stepci/plugin-openapi/issues
Disclaimer: We have tested our OpenAPI integration with the Petstore example and some other API definitions we had available. In both cases the generated tests would run after only some little adjustments
Really cool, with already a lot of features!
Given the raise of CI/CD, I really think these kind of tools (CLI tests on HTTP requests), based on a simple format, will be really important. We've build Hurl (https://github.com/Orange-OpenSource/hurl), that shares a lot of similarities with Step CI (plain text instead of yaml, captures, jsonpath, xpath etc...). I will shamelessly take inspiration for some new features (like GraphQL for instance)!
Oh, nice to meet you!
Our Matcher functionality was actually inspired by Hurl (you call them predicates!)! We would take your inspiration from us as a huge compliment :)
The main difference between Hurl and Step CI is from our persepective an ideological one. What do you think?
Hurl is a CLI tool, while Step CI is a library. This means that you can include our Test Runner (https://github.com/stepci/runner) and use it with other tools. For example, you can combine it with testing frameworks like Mocha, Jest, and Ava (https://github.com/stepci/stepci/blob/main/docs/using-test-t...) or build your own platform on top of it.
That's what's so nice about open source.
From my experience, generated tests are worthless for anything more serious than smoke tests. I prefer working with no tests than automated tests, I feel they give you a false sense of confidence.
The Step CI engine itself looks good though. It looks like a cleaner, but less powerful version of a tool (open source, build in-house) we used when I worked at OVHcloud, Venom: https://github.com/ovh/venom
Here's an example test file for the HTTP executor of Venom: https://github.com/ovh/venom/blob/master/tests/http.yml it's very close to Step CI format.
I'd still use Venom because it's way more powerful (you have DB executors for example, so after executing a POST request you can actually check in DB that you have what you expect) and I prefer focusing on actually writing integration tests instead of generating them.
Maybe this post sounds harsh (I feel it as I write it because I have strong feelings against test generation) but I think your approach is a good one for actually writing automated tests. Testing APIs declaratively like this has a great benefit: your tests work on an interface. You can migrate your API to a whole new stack and your tests remain the same. I did it multiple time at OVHcloud: one time migrating a huge API from a Go router to another (Gin->Echo), and another time migrating public APIs from a legacy, in-house Perl engine to a Go server.