google-research VS brotli

People on here will be happy to say that I do a similar thing, however my sequence length is dynamic because I also use a 2nd data structure - I'll use pretentious academic speak: I use a simple bigram LM (2-gram) for single next-word likeliness and separately a trie that models all words and phrases (so, n-gram). Not sure how many total nodes because sentence lengths vary in training data, but there are about 200,000 entry points (keys) so probably about 2-10 million total nodes in the default setup.

"Constructing 7-gram LM": They likely started with bigrams (what I use) which only tells you the next word based on 1 word given, and thought to increase accuracy by modeling out more words in a sequence, and eventually let the user (developer) pass in any amount they want to model (https://github.com/google-research/google-research/blob/5c87...). I thought of this too at first, but I actually got more accuracy (and speed) out of just keeping them as bigrams and making a totally separate structure that models out an n-gram of all phrases (e.g. could be a 24-token long sequence or 100+ tokens etc. I model it all) and if that phrase is found, then I just get the bigram assumption of the last token of the phrase. This works better when the training data is more diverse (for a very generic model), but theirs would probably outperform mine on accuracy when the training data has a lot of nearly identical sentences that only change wildly toward the end - I don't find this pattern in typical data though, maybe for certain coding and other tasks there are those patterns though. But because it's not dynamic and they make you provide that number, even a low number (any phrase longer than 2 words) - theirs will always have to do more lookup work than with simple bigrams and they're also limited by that fixed number as far as accuracy. I wonder how scalable that is - if I need to train on occasional ~100-word long sentences but also (and mostly) just ~3-word long sentences, I guess I set this to 100 and have a mostly "undefined" trie.

I also thought of the name "LMJS", theirs is "jslm" :) but I went with simply "next-token-prediction" because that's what it ultimately does as a library. I don't know what theirs is really designed for other than proving a concept. Most of their code files are actually comments and hypothetical scenarios.

I recently added a browser example showing simple autocomplete using my library: https://github.com/bennyschmidt/next-token-prediction/tree/m... (video)

And next I'm implementing 8-dimensional embeddings that are converted to normalized vectors between 0-1 to see if doing math on them does anything useful beyond similarity, right now they look like this:

  [nextFrequency, prevalence, specificity, length, firstLetter, lastLetter, firstVowel, lastVowel]

The change was literally just made: https://github.com/google-research/google-research/commit/4a...

It appears this was in response to Hacker News comments.

There are options such as Google's ScaNN that may let you go farther before needing to consider specialized databases.

https://github.com/google-research/google-research/blob/mast...

I feel it was unnecesary to create this because https://research.google/ already exists? It just seems like they want to take another URL with a "pure" domain name instead of psubdirectories, etc parts.

https://github.com/google-research/google-research/blob/mast...

You could also try some dedicated translation models like https://huggingface.co/facebook/nllb-moe-54b (or https://github.com/google-research/google-research/tree/master/madlad_400 for something smaller) and see how they do.

Google released T5X checkpoints for MADLAD-400 a couple of months ago, but nobody could figure out how to run them. Turns out the vocabulary was wrong, but they uploaded the correct one last week.

Using tools like Brotli, you can boost your application’s load time. You can use the ngUpgrade library to mix AngularJS and Angular components to enhance runtime performance, bringing in hybrid applications that can be used with techniques like ahead-of-time (AOT) compilation, aiding in faster browser rendering.

JPEGLI = A small JPEG

The suffix -li is used in Swiss German dialects. It forms a diminutive of the root word, by adding -li to the end of the root word to convey the smallness of the object and to convey a sense of intimacy or endearment.

This obviously comes out of Google Zürich.

Other notable Google projects using Swiss German:

https://github.com/google/gipfeli high-speed compression

Gipfeli = Croissant

https://github.com/google/guetzli perceptual JPEG encoder

Guetzli = Cookie

https://github.com/weggli-rs/weggli semantic search tool

Weggli = Bread roll

https://github.com/google/brotli lossless compression

Brötli = Small bread

The brotli repo on github has a dictionary generator: https://github.com/google/brotli/blob/master/research/dictio...

I have a hosted version of it on https://use-as-dictionary.com/ to make it easier to experiment with.

An additional element that we can finally remove from our stack is the minification of JavaScript and CSS files. Thanks to algorithms like brotli (with a very Swiss flavour) we no longer need to minify and compress our files before distributing them. Cloudflare, Nginx, or Apache will take care of everything for us.

As you navigate the intricate world of JavaScript compression and web development, having a trusted partner by your side can make all the difference. That's where Coding Crafts comes in. At Coding Crafts, we take pride in being a top-tier software development company in USA. Our team of experts specializes in web development, optimization, and everything in between. As the best IT company in USA, we are dedicated to delivering cutting-edge solutions that drive performance and efficiency. Our expertise extends to choosing the right compression technique for your web application, ensuring that your website performs optimally. In conclusion, the choice of JavaScript compression technique depends on various factors, including your specific goals, browser and server support, and performance requirements. Whether you opt for Brotli, GZIP, or Zopfli, Coding Crafts is here to provide the guidance and expertise you need to enhance your web application's performance and user experience. For more information on how Coding Crafts can assist you with your web development and optimization needs, contact us today. Resources "Brotli - GitHub Repository": https://github.com/google/brotli "Zopfli - Google Developers": https://developers.google.com/speed/articles/zopfli "Introduction to GZIP Compression - MDN Web Docs": https://developer.mozilla.org/en-US/docs/Web/HTTP/Overview#gzip_compression "Brotli vs. GZIP vs. Zopfli: Which Compression Method is Best?" - KeyCDN Blog: https://www.keycdn.com/blog/brotli-vs-gzip-vs-zopfli

We serve your site from a global cache location close to your visitors to make sure your site loads fast. In addition, we use an advanced HTML and text compression algorithm called Brotli. Compressed content is now cached, so we can send it directly to your visitors instead of compressing each request individually. In our tests this often improves loading speed by up to 2x, which will have a very positive impact on your Lighthouse scores like LCP. This will be especially noticeable on larger sites, so you can scale your site without worry.

For anyone who wants to try this, zstd -T0 uses all your threads to compress, and https://github.com/facebook/zstd has a lot more description. Brotli, https://github.com/google/brotli, is another modern format with some good features for high compression levels and Content-Encoding support in web browsers. You might also want to play with the compression level (-1 to -11 or more, zstd's --fast=n).

One reason these modern compressors do better is not any particular mistake made defining DEFLATE in the 90s, but that new algos use a few MB of recently seen data as context instead of 32KB, and do other things impractical in the 90s but reasonable on modern hardware. The new algorithms also contain logs of smart ideas and have fine-tuned implementations, but that core difference seems important to note.

It seems to be using a lib called brotli - https://github.com/google/brotli. Can you compile from source?

(++) Play Youtube videos and playlistsRelease date: Feb 3 20223dyd, [email protected] libraries: brotli https://github.com/google/brotli (1.0.7)

Alright, the minifying is done. What else? Did you know you can serve HTML, CSS and JS compressed? A lot of websites still use gzip, but there’s also Brotli. Brotli is specifically made for the web and compresses a lot better than gzip in most cases.