transformers VS tailwind-nextjs-starter-blog

Is t5x an encoder/decoder architecture?

Some more general options.

The Flax ecosystem

https://github.com/google/flax?tab=readme-ov-file

or dm-haiku

https://github.com/google-deepmind/dm-haiku

were some of the best developed communities in the Jax AI field

Perhaps the “trax” repo? https://github.com/google/trax

Some HF examples https://github.com/huggingface/transformers/tree/main/exampl...

Sadly it seems much of the work is proprietary these days, but one example could be Grok-1, if you customize the details. https://github.com/xai-org/grok-1/blob/main/run.py

The HuggingFace transformers library already has support for a similar method called prompt lookup decoding that uses the existing context to generate an ngram model: https://github.com/huggingface/transformers/issues/27722

I don't think it would be that hard to switch it out for a pretrained ngram model.

Most of this tutorial is based on Hugging Face course about Transformers and on Niels Rogge's Transformers tutorials: make sure to check their work and give them a star on GitHub, if you please ❤️

* Superconvergence + LR range finder + Fast AI's Ranger21 optimizer was the goto optimizer for CNNs, and worked fabulously well, but on transformers, the learning rate range finder sadi 1e-3 was the best, whilst 1e-5 was better. However, the 1 cycle learning rate stuck. https://github.com/huggingface/transformers/issues/16013

Thanks! :) I'm pushing them into transformers, pytorch-gemma and collabing with the Gemma team to resolve all the issues :)

The RoPE fix should already be in transformers 4.38.2: https://github.com/huggingface/transformers/pull/29285

My main PR for transformers which fixes most of the issues (some still left): https://github.com/huggingface/transformers/pull/29402

If you want to tinker with the architecture Hugging Face has a FOSS implementation in transformers: https://github.com/huggingface/transformers/blob/main/src/tr...

If you want to reproduce the training pipeline, you couldn't do that even if you wanted to because you don't have access to thousands of A100s.

I am aware that using mixed precision reduces the stability of weight and there will be little consistency but don't expect it to be this much. I have attached the graph of evaluation metrics. If someone can give me some insight into this issue, that would be great.

Inspired by tailwind-nextjs-starter-blog

At Axolo (https://axolo.co/blog), we love this tailwind next.js open source project https://github.com/timlrx/tailwind-nextjs-starter-blog

I recommend that you find a template that looks reasonably good. You can than spend time on adding features to an already nice looking page. this is the template that I started with: https://tailwind-nextjs-starter-blog.vercel.app/ and I have made some small tweaks over time and now it looks like this: https://www.einargudni.com/

The repo you linked uses this pattern: https://github.com/timlrx/tailwind-nextjs-starter-blog/tree/master/pages/blog

Also visit the link below, in the answer there's a website that will help you https://github.com/timlrx/tailwind-nextjs-starter-blog/issues/577

I'm a fan of timlrx/tailwind-nextjs-start-blog [1]. It's easy to customize, responsive, uses Markdown, and makes use of SSR for SEO optimization.

https://github.com/timlrx/tailwind-nextjs-starter-blog#quick...

You can deploy it on Vercel for free and then point it to a custom domain using DNS records in the Vercel "Domains" tab.

I've been looking for a good, flexible template for SvelteKit / Tailwind CSS, but all the templates I found were either too simple, missing some needed features, or were too difficult to customize (or both). I loved Timothy Lin's simple but feature-rich Tailwind Nextjs Started Blog, so I decided to port that to SvelteKit, and add a few things I needed, including Netlify CMS, dynamic Open Graph image (og:image) support.