jsverify
venom
jsverify | venom | |
---|---|---|
5 | 6 | |
1,666 | 974 | |
0.1% | 2.9% | |
1.8 | 7.2 | |
about 3 years ago | 7 days ago | |
JavaScript | Go | |
MIT License | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
jsverify
-
The 5 principles of Unit Testing
Libraries like JSVerify or Fast-Check offer essential tools to facilitate property-based testing.
-
Ask HN: What's your favorite software testing framework and why?
I tend to use anything that offers property-testing, since tests are much shorter to write and uncover lots more hidden assumptions.
My go-to choices per language are:
- Python: Hypothesis https://hypothesis.readthedocs.io/en/latest (also compatible with PyTest)
- Scala: ScalaCheck https://scalacheck.org (also compatible with ScalaTest)
- Javascript/Typescript: JSVerify https://jsverify.github.io
- Haskell: LazySmallCheck2012 https://github.com/UoYCS-plasma/LazySmallCheck2012/blob/mast...
- When I wrote PHP (over a decade ago) there was no decent property-based test framework, so I cobbled one together https://github.com/Warbo/php-easycheck
All of the above use the same basic setup: tests can make universally-quantified statements (e.g. "for all (x: Int), foo(x) == foo(foo(x))"), then the framework checks that statement for a bunch of different inputs.
Most property-checking frameworks generate data randomly (with more or less sophistication). The Haskell ecosystem is more interesting:
- QuickCheck was one of the first property-testing frameworks, using random genrators.
- SmallCheck came later, which enumerates data instead (e.g. testing a Float might use 0, 1, -1, 2, -2, 0.5, -0.5, etc.). That's cute, but QuickCheck tends to exercise more code paths with each input.
- LazySmallCheck builds up test data on-demand, using Haskell's pervasive laziness. Tests are run with an error as input: if they pass, we're done; if they fail, we're done; if they trigger the error, they're run again with slightly more-defined inputs. For example, if the input is supposed to be a list, we try again with the two forms of list: empty and "cons" (the arguments to cons are both errors, to begin with). This exercises even more code paths for each input.
- LazySmallCheck2012 is a more versatile "update" to LazySmallCheck; in particular, it's able to generate functions.
-
Property Based Testing Framework for Node
The usage of hypothesis is very intuitive and simple, and presents the concept of property-based testing perfectly. So I also wanted to find an equivalent alternative in Node. Two of them have high star ratings on Github, JSVerify with 1.6K stars and fast-check with 2.8K stars. So I took some time to study fast-check a little bit and try to get closer to my daily work.
-
Machine Readable Specifications at Scale
Systems I've used for this include https://agda.readthedocs.io/en/v2.6.0.1/getting-started/what... https://coq.inria.fr https://www.idris-lang.org and https://isabelle.in.tum.de
An easier alternative is to try disproving the statement, by executing it on thousands of examples and seeing if any fail. That gives us less confidence than a full proof, but can still be better than traditional "there exists" tests. This is called property checking or property-based testing. Systems I've used for this include https://hypothesis.works https://hackage.haskell.org/package/QuickCheck https://scalacheck.org and https://jsverify.github.io
-
React to Elm Migration Guide
Using create-react-app, you’ll run npm test which uses Jest internally. If you are dealing with a lot of data on the UI, or using TypeScript, use JSVerify for property tests. For end to end tests, Cypress is a great choice.
venom
-
Ask HN: What's your favorite software testing framework and why?
You can also load fixtures in database directly, work with Kafka queues both as a producer (e.g. write an event to a Kafka queue, wait a few seconds and see that it was consumed by the service you test, and that some side effects can be observed) or as a consumer (e.g. make sure after an HTTP call, an event was correctly pushed to a queue), or even read a mailbox in IMAP to check that your service correctly send an email.
It's a bit rough on the edges sometimes, but I'd never go back on writing integration tests directly in my programming language. Declarative is the way to go.
[1]: https://github.com/ovh/venom
-
Easy Integration Testing with Venom!
To write and run our integration tests, we'll use Venom. Venom is a tool created and made open-source by OVHcloud: https://github.com/ovh/venom
- Venom: Manage and run your integration tests with efficiency
-
Show HN: Step CI – API Testing and Monitoring Made Simple
From my experience, generated tests are worthless for anything more serious than smoke tests. I prefer working with no tests than automated tests, I feel they give you a false sense of confidence.
The Step CI engine itself looks good though. It looks like a cleaner, but less powerful version of a tool (open source, build in-house) we used when I worked at OVHcloud, Venom: https://github.com/ovh/venom
Here's an example test file for the HTTP executor of Venom: https://github.com/ovh/venom/blob/master/tests/http.yml it's very close to Step CI format.
I'd still use Venom because it's way more powerful (you have DB executors for example, so after executing a POST request you can actually check in DB that you have what you expect) and I prefer focusing on actually writing integration tests instead of generating them.
Maybe this post sounds harsh (I feel it as I write it because I have strong feelings against test generation) but I think your approach is a good one for actually writing automated tests. Testing APIs declaratively like this has a great benefit: your tests work on an interface. You can migrate your API to a whole new stack and your tests remain the same. I did it multiple time at OVHcloud: one time migrating a huge API from a Go router to another (Gin->Echo), and another time migrating public APIs from a legacy, in-house Perl engine to a Go server.
-
Debugging with GDB
I still struggle with GDB but my excuse is that I seldom use it.
When I was studying reverse engineering though, I came across a really cool kit (which I've yet to find an alternative for lldb, which would be nice given: rust)
I'd recommend checking it out, if for no other reason than it makes a lot of things really obvious (like watching what value lives in which register).
https://github.com/hugsy/gef
LLDB's closest alternative to this is called Venom, but it's not the same at all. https://github.com/ovh/venom
-
Do you write integration tests in go?
We incorporated [Venom](https://github.com/ovh/venom) into our workflow. It's great for initiating and managing a suite of yaml based tests. It didn't work out of the box for us due to the heavily asynchronous nature of our system, but after a few additions, it has helped my team greatly. We were often afraid to make large changes to critical pieces of the system since a full regression test could take a week or so to check everything. Now it takes an hour.
What are some alternatives?
greenlight - Clojure integration testing framework
godog - Cucumber for golang
testy - test helpers for more meaningful, readable, and fluent tests
dockertest - Write better integration tests! Dockertest helps you boot up ephermal docker images for your Go tests with minimal work.
LazySmallCheck2012 - Lazy SmallCheck with functional values and existentials!
testcontainers-go - Testcontainers for Go is a Go package that makes it simple to create and clean up container-based dependencies for automated integration/smoke tests. The clean, easy-to-use API enables developers to programmatically define containers that should be run as part of a test and clean up those resources when the test is done.
fast-check - Property based testing framework for JavaScript (like QuickCheck) written in TypeScript
stepci - Automated API Testing and Quality Assurance
hitchstory - Type-safe YAML integration tests. Tests that write your docs. Tests that rewrite themselves.
gotestfmt - go test output for humans
datadriven - Data-Driven Testing for Go
gotestfmt - go test output for humans