jepsen
ansi-test
Our great sponsors
jepsen | ansi-test | |
---|---|---|
3 | 2 | |
6,603 | 14 | |
1.2% | - | |
7.6 | 2.8 | |
about 1 month ago | about 1 year ago | |
Clojure | Common Lisp | |
- | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
jepsen
-
MongoDB is actively investigating a security incident
aphyr seems to still work on it, so probably just not that many companies who want to pay to get told the truth anymore: https://github.com/jepsen-io/jepsen
-
What Is Property Based Testing?
One very nice example of this is Jepsen:
https://github.com/jepsen-io/jepsen
-
ZooKeeper-free Kafka is out. First Demo
https://github.com/jepsen-io/jepsen/issues/399
> Yeah! I mean, I find a lot of linearizability errors in various databases, but this was also my very first time doing this kind of test, and it varies from system to system. Could have easily slipped through the cracks.
In summary, aphyr thought Zookeeper is linearizable even though it doesn't provide linearizable ops.
Looks like Zookeeper needs to be tested again.
ansi-test
- ECL could run as fast as SBCL if it does more type inference and aggressive function inlining
-
What Is Property Based Testing?
I have applied it to testing Common Lisp implementations, but there has been much work on applying it to other languages (most famously C and Javascript.)
I applied the following techniques:
(1) Generate random valid well defined programs and see (a) if they crash the compiler, (b) cause different CL implementations to produce different outputs, (c) when modified (by addition of randomly generated optimization directives or valid type declarations) they still produce the same output. This is differential testing, which was used by McKeeman at DEC in the 1990s to test C compilers, and later improved (again, on C compilers) by Yang, Chan, Eide, and Regehr at U. of Utah (csmith, creduce).
Since a running lisp image can generate and compile functions internally (this IS lisp, after all), the testing loop can be very fast. Since 2003 I have run this on and off for many billions of iterations on desktop and laptop machines on various CL implementations, now mainly on SBCL. Most of the test input reduction is handled automatically, which is a big help.
(2) Generate random possibly invalid code by mutating or recombining snippets drawn from a large corpus of code, to see if it crashes the compiler (in CL implementations where the compiler is promised to never respond to bad code by signaling an error.) This was also the approach jsfunfuzz took on Javascript.
(3) Extensive fuzzing of calls to standard functions in CL, using random generation of input values and random generation of valid type declarations, with the invariant being that the same values should be computed (and the compiler not fail.) This is a specialization of (1), but was sufficiently different that the bugs it found were not the same.
Examples of tests produced by (1) and (3) over an early period when this was being developed. Each caused a failure in some CL implementation: https://github.com/pfdietz/ansi-test/blob/master/misc/misc.l...
See also the various bugs I've reported against SBCL over the years, many of which come from this testing. https://bugs.launchpad.net/~paul-f-dietz
The experience with this sort of testing of compilers (in any language) is that if the compiler (free or commercial) has never been subjected to it, it will immediately find bugs in the compiler.
What are some alternatives?
jetstream - JetStream Utilities
Schemathesis - Automate your API Testing: catch crashes, validate specs, and save time
java-pubsublite-kafka
redpanda - Redpanda is a streaming data platform for developers. Kafka API compatible. 10x faster. No ZooKeeper. No JVM!
Apache Pulsar - Apache Pulsar - distributed pub-sub messaging system
Apache Kafka - Mirror of Apache Kafka
kafkacat - Generic command line non-JVM Apache Kafka producer and consumer [Moved to: https://github.com/edenhill/kcat]
liftbridge - Lightweight, fault-tolerant message streams.
Jocko - Kafka implemented in Golang with built-in coordination (No ZK dep, single binary install, Cloud Native)
hypothesis - Hypothesis is a powerful, flexible, and easy to use library for property-based testing.
styx - Simple, high-performance event streaming broker