brotli VS LZ4

An additional element that we can finally remove from our stack is the minification of JavaScript and CSS files. Thanks to algorithms like brotli (with a very Swiss flavour) we no longer need to minify and compress our files before distributing them. Cloudflare, Nginx, or Apache will take care of everything for us.

For anyone who wants to try this, zstd -T0 uses all your threads to compress, and https://github.com/facebook/zstd has a lot more description. Brotli, https://github.com/google/brotli, is another modern format with some good features for high compression levels and Content-Encoding support in web browsers. You might also want to play with the compression level (-1 to -11 or more, zstd's --fast=n).

One reason these modern compressors do better is not any particular mistake made defining DEFLATE in the 90s, but that new algos use a few MB of recently seen data as context instead of 32KB, and do other things impractical in the 90s but reasonable on modern hardware. The new algorithms also contain logs of smart ideas and have fine-tuned implementations, but that core difference seems important to note.

It seems to be using a lib called brotli - https://github.com/google/brotli. Can you compile from source?

(++) Play Youtube videos and playlistsRelease date: Feb 3 20223dyd, [email protected] libraries: brotli https://github.com/google/brotli (1.0.7)

Alright, the minifying is done. What else? Did you know you can serve HTML, CSS and JS compressed? A lot of websites still use gzip, but there’s also Brotli. Brotli is specifically made for the web and compresses a lot better than gzip in most cases.

You can use CloudFront to automatically compress certain types of objects (files) and serve the compressed objects when viewers (web browsers or other clients) support them. Viewers indicate their support for compressed objects with the Accept-Encoding HTTP header. CloudFront can compress objects using the Gzip and Brotli compression formats. When the viewer supports both formats, CloudFront prefers Brotli.

Hi /r/java! Jetty is considering implementing dynamic Brotli compression, but the current JVM wrapper for Google's Brotli (jvm-brotli) is somewhat ... abandoned.

Year 2022 marked an amazing start for me with a new stint with epilot GmbH based out of Cologne, Germany. I joined the team to be working on their micro-frontend architecture, with my primary focus going to be on the performance aspects of the applications. Like the first recent addition we did after I started was to add brotli compression to each of our micro-frontend (MFEs) bundles.

The actual vulnerability is in the third party brotli library. The patch that fixes the vulnerability is here https://github.com/google/brotli/releases/tag/v1.0.9

According to these https://github.com/lz4/lz4 values you need around ten (10) quite modern cores in parallel to accomplish around 8GB/s.

> The downside of lz4 is that it can’t be configured to run at higher & slower compression ratios.

lz4 has some level of configurability? https://github.com/lz4/lz4/blob/v1.9.4/lib/lz4frame.h#L194

There's also LZ4_HC.

Get your bootloader unlocked via Download mode and then obtain your stock firmware, preferably for your current region https://samfw.com (Download mode: CARRIER_CODE). Get the boot image from AP with 7zip, unpack from LZ4 with https://github.com/lz4/lz4/releases (drag and drop), patch with Magisk https://github.com/topjohnwu/magisk/releases/latest, grab the new image, name it "boot.img" and pack it into a .tar with 7zip and flash to AP with odin https://odindownload.com

After some investigations and experiments, I found out that it was the PNG compression (well, decompression I should say) that took a while. So I've made some experiments using the LZ4 compression library, which is focused on decompression speed, and it turned out to be an excellent solution!

If anyone just cares for speed instead of compression I’d recommend lz4 [1]. I only recently started using it. Its speed is almost comparable to memcpy.

[1] https://github.com/lz4/lz4

E: Realtime compression (A good compression library like Zstandard can make a game less than half the size while taking a tiny amount of CPU power when loading stuff. I think thats a pretty worthwhile trade.) (ZSTD github) (LZ4 github)

lz4 for maximum decompression speed, for data that is often read but rarely written

Getting lost in material models is tempting. But, at this point you are overdue for working on your own asset pipeline. glTF is great. But, you should learn how to do it yourself. The hardest part will be reading source asset files. The FBX SDK is painful. Assimp isn't great either. Writing your own exporter to your own intermediate text format from Maya or Blender would be good if you are up for it. From whatever source, make your own archive format and binary formats for meshes, animations, textures and scenes. Use https://github.com/lz4/lz4 for compression. You should be able to decompress a list of assets into a big linear array and use them right there with just a bit of pointer fix-up. Minimize the amount of memory you have to touch from start to finish. Data that is going to the GPU (textures, vertex/index buffers) should decompress straight into mapped buffers for fast uploads.

I'm not a fan of the stacked bar charts, I like the table of data for "Benchmarks" on the github source page: https://github.com/lz4/lz4

It makes it very clear where LZ4 fits into comparisons with compression speed, decompression speed and compression ratio