LazySmallCheck2012
venom
LazySmallCheck2012 | venom | |
---|---|---|
2 | 6 | |
4 | 979 | |
- | 2.1% | |
0.0 | 7.3 | |
4 months ago | 12 days ago | |
Haskell | Go | |
BSD 3-clause "New" or "Revised" License | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
LazySmallCheck2012
-
Generating Well-Typed Terms that are not "Useless" [pdf]
Using laziness to avoid generating parts of an expression until it's needed is a really nice idea. The LazySmallCheck package[1] took this approach, but was limited in the types of data it could produce (e.g. it couldn't generate functions). This was extended by LazySmallCheck2012[2], but that seems to be unmaintained and doesn't work with more recent GHC versions.
(Note that these are named in reference to SmallCheck[3], which takes the approach of enumerating concrete values in order of "size"; as an alternative to the more widely-used QuickCheck[4], which generates concrete values at random, and tries to "shrink" those which trigger a failure)
[1] https://hackage.haskell.org/package/lazysmallcheck
[2] https://github.com/UoYCS-plasma/LazySmallCheck2012
[3] https://hackage.haskell.org/package/smallcheck
[4] https://hackage.haskell.org/package/QuickCheck
-
Ask HN: What's your favorite software testing framework and why?
I tend to use anything that offers property-testing, since tests are much shorter to write and uncover lots more hidden assumptions.
My go-to choices per language are:
- Python: Hypothesis https://hypothesis.readthedocs.io/en/latest (also compatible with PyTest)
- Scala: ScalaCheck https://scalacheck.org (also compatible with ScalaTest)
- Javascript/Typescript: JSVerify https://jsverify.github.io
- Haskell: LazySmallCheck2012 https://github.com/UoYCS-plasma/LazySmallCheck2012/blob/mast...
- When I wrote PHP (over a decade ago) there was no decent property-based test framework, so I cobbled one together https://github.com/Warbo/php-easycheck
All of the above use the same basic setup: tests can make universally-quantified statements (e.g. "for all (x: Int), foo(x) == foo(foo(x))"), then the framework checks that statement for a bunch of different inputs.
Most property-checking frameworks generate data randomly (with more or less sophistication). The Haskell ecosystem is more interesting:
- QuickCheck was one of the first property-testing frameworks, using random genrators.
- SmallCheck came later, which enumerates data instead (e.g. testing a Float might use 0, 1, -1, 2, -2, 0.5, -0.5, etc.). That's cute, but QuickCheck tends to exercise more code paths with each input.
- LazySmallCheck builds up test data on-demand, using Haskell's pervasive laziness. Tests are run with an error as input: if they pass, we're done; if they fail, we're done; if they trigger the error, they're run again with slightly more-defined inputs. For example, if the input is supposed to be a list, we try again with the two forms of list: empty and "cons" (the arguments to cons are both errors, to begin with). This exercises even more code paths for each input.
- LazySmallCheck2012 is a more versatile "update" to LazySmallCheck; in particular, it's able to generate functions.
venom
-
Ask HN: What's your favorite software testing framework and why?
You can also load fixtures in database directly, work with Kafka queues both as a producer (e.g. write an event to a Kafka queue, wait a few seconds and see that it was consumed by the service you test, and that some side effects can be observed) or as a consumer (e.g. make sure after an HTTP call, an event was correctly pushed to a queue), or even read a mailbox in IMAP to check that your service correctly send an email.
It's a bit rough on the edges sometimes, but I'd never go back on writing integration tests directly in my programming language. Declarative is the way to go.
[1]: https://github.com/ovh/venom
-
Easy Integration Testing with Venom!
To write and run our integration tests, we'll use Venom. Venom is a tool created and made open-source by OVHcloud: https://github.com/ovh/venom
- Venom: Manage and run your integration tests with efficiency
-
Show HN: Step CI – API Testing and Monitoring Made Simple
From my experience, generated tests are worthless for anything more serious than smoke tests. I prefer working with no tests than automated tests, I feel they give you a false sense of confidence.
The Step CI engine itself looks good though. It looks like a cleaner, but less powerful version of a tool (open source, build in-house) we used when I worked at OVHcloud, Venom: https://github.com/ovh/venom
Here's an example test file for the HTTP executor of Venom: https://github.com/ovh/venom/blob/master/tests/http.yml it's very close to Step CI format.
I'd still use Venom because it's way more powerful (you have DB executors for example, so after executing a POST request you can actually check in DB that you have what you expect) and I prefer focusing on actually writing integration tests instead of generating them.
Maybe this post sounds harsh (I feel it as I write it because I have strong feelings against test generation) but I think your approach is a good one for actually writing automated tests. Testing APIs declaratively like this has a great benefit: your tests work on an interface. You can migrate your API to a whole new stack and your tests remain the same. I did it multiple time at OVHcloud: one time migrating a huge API from a Go router to another (Gin->Echo), and another time migrating public APIs from a legacy, in-house Perl engine to a Go server.
-
Debugging with GDB
I still struggle with GDB but my excuse is that I seldom use it.
When I was studying reverse engineering though, I came across a really cool kit (which I've yet to find an alternative for lldb, which would be nice given: rust)
I'd recommend checking it out, if for no other reason than it makes a lot of things really obvious (like watching what value lives in which register).
https://github.com/hugsy/gef
LLDB's closest alternative to this is called Venom, but it's not the same at all. https://github.com/ovh/venom
-
Do you write integration tests in go?
We incorporated [Venom](https://github.com/ovh/venom) into our workflow. It's great for initiating and managing a suite of yaml based tests. It didn't work out of the box for us due to the heavily asynchronous nature of our system, but after a few additions, it has helped my team greatly. We were often afraid to make large changes to critical pieces of the system since a full regression test could take a week or so to check everything. Now it takes an hour.
What are some alternatives?
jsverify - Write powerful and concise tests. Property-based testing for JavaScript. Like QuickCheck.
godog - Cucumber for golang
testy - test helpers for more meaningful, readable, and fluent tests
dockertest - Write better integration tests! Dockertest helps you boot up ephermal docker images for your Go tests with minimal work.
tricorder - Automation the KISS way
testcontainers-go - Testcontainers for Go is a Go package that makes it simple to create and clean up container-based dependencies for automated integration/smoke tests. The clean, easy-to-use API enables developers to programmatically define containers that should be run as part of a test and clean up those resources when the test is done.
ospec - Noiseless testing framework
stepci - Automated API Testing and Quality Assurance
datadriven - Data-Driven Testing for Go
gotestfmt - go test output for humans
smallcheck - Test your Haskell code by exhaustively checking its properties
gotestfmt - go test output for humans