jmh VS Jimfs

JMH for benchmarks

I would recommend using JMH instead.

Reflection is probably very roughly between 10 and 1000 times slower. Why don't you measure it yourself using JMH?

Aleksey Shipilev is another such leader, whom is especially knowledgeable about the internals of the JVM. His writings are invaluable. He is (was) the lead of the Java microbenchmark framework (JMH} which is how one would write small performance experiments in Java, and learn what really makes a difference or now.

In the last episode we implemented a parallelMap operation using streams, raw threads, a threadpool with futures, and coroutines. At first glance the raw threads was quickest, followed by futures, coroutines and then streams. In this, part 56 of an exploration of where a Test Driven Development implementation of the Gilded Rose stock control system might take us in Kotlin, we investigate the performance of the different functions further, in particular digging down into why coroutines seem to be slow and finding a way to speed them up. We also find a way to use a particular ForkJoinPool to run the streams code, making it as fast as the others (bar the raw threads). Frankly we only use very rough benchmarks here, with no statistical testing except 'it looks like'. That's OK for gross differences, but is highly suspect when deciding which of two similarly performant approaches is faster. For that check out JMH and you could watch my video from KotlinConf 2017

A few months ago, we started to use JMH in our project to test and find performance issues. The tool provides multiple modes and profilers, and we found this useful for our purposes.

Java has in-memory file systems that are essentially geared for this exact use case, eg jimfs[0]. You create your filesystem and any files you need when your tests are starting up, and your classes talk to them rather than the “real” ones. Maybe a similar project exists for the C++ ecosystem?

[0] https://github.com/google/jimfs

Recently been using JIMFS. Made my tests much faster and cleaner!

I use Google's JIMFS "Just In Memory Filesystem" https://github.com/google/jimfs in my unit tests and have been very happy. No need to clean something up that disappears as soon as the test is over. Let's you create unix or windows style filesystems and I've used it to test a disk space healthcheck because you can set a limit to the size of the filesystem it creates. Very flexible and easy to use.