smaz VS brotli

Choose the data compression algorithm based on the specifics of your data. For example, if you are working with lots of short strings, take a look at [*SMAZ](https://github.com/antirez/smaz).*

Take a look at this. Idea behind it seems nice, but it's fixed dictionary ("codebook") was clearly made for English language, and the algorithm itself is really simple. How can we impove on this? Dynamic dictionary won't do, since you have to store it somewhere, nullifying benefits of using such algorithm. Beyond that I have no idea.

smaz - Efficient string compression library. BSD-3-Clause

We recommend pre-compressing assets during your build pipeline using the official https://github.com/google/brotli CLI tool, which allows you to use higher compression levels (9-11) without adding latency to requests. Our benchmarks show that pre-compressed brotli assets at level 11 are 22% smaller than on-the-fly brotli level 6, with zero runtime overhead. For teams serving more than 100GB of static assets monthly, this optimization alone can save $500+ in bandwidth costs. Always validate compression ratios with the https://github.com/caddyserver/caddy built-in debug logs or Nginx’s $brotli_ratio variable to ensure your compression settings are effective. This tip is especially valuable for teams with mobile-heavy user bases, where reduced payload sizes directly correlate to lower bounce rates and higher conversion.

Modern web format is woff (Web Open Font Format) with ~97% browser support. Version 2 is using Brotli compression and is ~20% - 50% more efficient.

Projects like Brotli aren't built to maximize personal profit; they're driven by passion and a genuine love for software engineering.

It's clear that the industry is shifting from being geeky and nerdy to being more business and management focused.

[0] https://github.com/google/brotli

Brotli is a compression algorithm developed by Google, particularly effective for text and web compression. It uses a combination of LZ77 (Lempel-Ziv 77), Huffman coding, and 2nd order context modeling. In comparison to traditional algorithms like Gzip, Brotli can achieve smaller compressed sizes, especially for HTML and text-heavy content. This makes it a good candidate for our Pied Piper-inspired text compression implementation.

The one example I can think of with a pre-seeded dictionary (for web, no less) is Brotli.

https://datatracker.ietf.org/doc/html/rfc7932#appendix-A

You can more or less see what it looks like (per an older commit): https://github.com/google/brotli/blob/5692e422da6af1e991f918...

Certainly it performs better than gzip by itself.

Some historical discussion: https://news.ycombinator.com/item?id=19678985

I believe the compression dictionary refers to [1], which is used to quickly match dictionary-compressable byte sequences. I don't know where 170 KB comes from, but that hash alone does take 128 KiB and might be significant if it can't be easily recomputed. But I'm sure that it can be quickly computed on the loading time if the binary size is that important.

[1] https://github.com/google/brotli/blob/master/c/enc/dictionar...

The answer lies in the Accept-Encoding header. In the example above, I just copied it from my browser, so it lists all the compression methods my browser supports: "gzip, deflate, br, zstd". The Wayfair backend supports compression with "br", which is Brotli, and uses it as the most efficient method.

...though with the slightly unexpected side effect (for Brotli, at least) that your executable may end up containing (~200KB, from memory) of very unexpected plain text strings which might (& has[0]) lead to questions from software end-users asking why your software contains "random"[1] text (including potentially "culturally sensitive" words/phrases related to religion such as "Holy Roman Emperor", "Muslims", "dollars", "emacs"[2] or similar).

(I encountered this aspect while investigating potential size optimization opportunities for the Godot game engine's web/WASM builds--though presumably the Brotli dictionary compresses well if the transfer encoding is... Brotli. :D )

[0] "This needs to be reviewed immediately #876": https://github.com/google/brotli/issues/876

[1] Which, regardless of meaning, certainly bears similarities to the type of "unexpected weird text" commonly/normally associated with spam, malware, LLMs and other entities of ill repute.

[2] The final example may not actually be factual. :)

Using tools like Brotli, you can boost your application’s load time. You can use the ngUpgrade library to mix AngularJS and Angular components to enhance runtime performance, bringing in hybrid applications that can be used with techniques like ahead-of-time (AOT) compilation, aiding in faster browser rendering.

JPEGLI = A small JPEG

The suffix -li is used in Swiss German dialects. It forms a diminutive of the root word, by adding -li to the end of the root word to convey the smallness of the object and to convey a sense of intimacy or endearment.

This obviously comes out of Google Zürich.

Other notable Google projects using Swiss German:

https://github.com/google/gipfeli high-speed compression

Gipfeli = Croissant

https://github.com/google/guetzli perceptual JPEG encoder

Guetzli = Cookie

https://github.com/weggli-rs/weggli semantic search tool

Weggli = Bread roll

https://github.com/google/brotli lossless compression

Brötli = Small bread