Snappy VS brotli

Another example of Nim being really fast is the supersnappy library. This library benchmarks faster than Google’s C or C++ Snappy implementation.

It doesn't destroy performance for the simple reason that nowadays memory access has higher latency than pure compute. If you need to use compute to produce some data to be stored in memory, your overall throughput could very well be faster than without compression.

There have been a large amount of innovation on fast compression in recent years. Traditional compression tools like gzip or xz are geared towards higher compression ratio, but memory compression tends to favor speed. Check out those algorithms:

* lz4: https://lz4.github.io/lz4/

* Google's snappy: https://github.com/google/snappy

* Facebook's zstd in fast mode: http://facebook.github.io/zstd/#benchmarks

An additional element that we can finally remove from our stack is the minification of JavaScript and CSS files. Thanks to algorithms like brotli (with a very Swiss flavour) we no longer need to minify and compress our files before distributing them. Cloudflare, Nginx, or Apache will take care of everything for us.

For anyone who wants to try this, zstd -T0 uses all your threads to compress, and https://github.com/facebook/zstd has a lot more description. Brotli, https://github.com/google/brotli, is another modern format with some good features for high compression levels and Content-Encoding support in web browsers. You might also want to play with the compression level (-1 to -11 or more, zstd's --fast=n).

One reason these modern compressors do better is not any particular mistake made defining DEFLATE in the 90s, but that new algos use a few MB of recently seen data as context instead of 32KB, and do other things impractical in the 90s but reasonable on modern hardware. The new algorithms also contain logs of smart ideas and have fine-tuned implementations, but that core difference seems important to note.

It seems to be using a lib called brotli - https://github.com/google/brotli. Can you compile from source?

(++) Play Youtube videos and playlistsRelease date: Feb 3 20223dyd, [email protected] libraries: brotli https://github.com/google/brotli (1.0.7)

Alright, the minifying is done. What else? Did you know you can serve HTML, CSS and JS compressed? A lot of websites still use gzip, but there’s also Brotli. Brotli is specifically made for the web and compresses a lot better than gzip in most cases.

You can use CloudFront to automatically compress certain types of objects (files) and serve the compressed objects when viewers (web browsers or other clients) support them. Viewers indicate their support for compressed objects with the Accept-Encoding HTTP header. CloudFront can compress objects using the Gzip and Brotli compression formats. When the viewer supports both formats, CloudFront prefers Brotli.

Hi /r/java! Jetty is considering implementing dynamic Brotli compression, but the current JVM wrapper for Google's Brotli (jvm-brotli) is somewhat ... abandoned.

Year 2022 marked an amazing start for me with a new stint with epilot GmbH based out of Cologne, Germany. I joined the team to be working on their micro-frontend architecture, with my primary focus going to be on the performance aspects of the applications. Like the first recent addition we did after I started was to add brotli compression to each of our micro-frontend (MFEs) bundles.

The actual vulnerability is in the third party brotli library. The patch that fixes the vulnerability is here https://github.com/google/brotli/releases/tag/v1.0.9