transformers VS redux-toolkit

Most of this tutorial is based on Hugging Face course about Transformers and on Niels Rogge's Transformers tutorials: make sure to check their work and give them a star on GitHub, if you please ❤️

* Superconvergence + LR range finder + Fast AI's Ranger21 optimizer was the goto optimizer for CNNs, and worked fabulously well, but on transformers, the learning rate range finder sadi 1e-3 was the best, whilst 1e-5 was better. However, the 1 cycle learning rate stuck. https://github.com/huggingface/transformers/issues/16013

Thanks! :) I'm pushing them into transformers, pytorch-gemma and collabing with the Gemma team to resolve all the issues :)

The RoPE fix should already be in transformers 4.38.2: https://github.com/huggingface/transformers/pull/29285

My main PR for transformers which fixes most of the issues (some still left): https://github.com/huggingface/transformers/pull/29402

If you want to tinker with the architecture Hugging Face has a FOSS implementation in transformers: https://github.com/huggingface/transformers/blob/main/src/tr...

If you want to reproduce the training pipeline, you couldn't do that even if you wanted to because you don't have access to thousands of A100s.

I am aware that using mixed precision reduces the stability of weight and there will be little consistency but don't expect it to be this much. I have attached the graph of evaluation metrics. If someone can give me some insight into this issue, that would be great.

In transformers, they tried really hard to have a single function or method to deal with both self and cross attention mechanisms, masking, positional and relative encodings, interpolation etc. While it allows a user to use the same function/method for any model, it has led to severe parameter bloat. Just compare the original implementation of llama by FAIR with the implementation by HF to get an idea.

pip install git+https://github.com/huggingface/transformers.git@main

In my example I am using Redux Toolkit and I got a prompt for actions to login and logout the user. If I need more functions, I can simply start typing the name, and Copilot provides the completion. For instance, in the example, I'm adding a function to update the user. And of course at the end of the file it suggests the exports.

Check out the official documentation.

Redux Toolkit 2.0

Redux Toolkit 2.0 is almost here! Hopefully shipping by this weekend :) Migration page

I am _thrilled_ to announce that:

Redux Toolkit 2.0 is LIVE!!!

- https://github.com/reduxjs/redux-toolkit/releases/tag/v2.0.0

This major version has new features, faster perf, smaller bundle size, and removes deprecated options.

It's accompanied by majors for all our Redux family packages

## RTK 2.0:

- a new `combineSlices` method for lazy-loading reducers - Updates to `createSlice` to include a `selectors` field and allow defining thunks inside

- Immer 10 w/ faster updates

- Removal of deprecated options

See the migration guide:

- https://redux.js.org/usage/migrations/migrating-rtk-2

All of the Redux libraries now have modernized packaging with full ESM/CJS compat. They also ship modern JS (no transpiling for IE11), which means smaller bundle sizes.

We've also done byte-shaving work to shrink the bundles (extracting error messages, de-duping imports)

## Redux core 5.0:

- The TS conversion we did in 2019!

- Action types _must_ be strings

- `UnknownAction` as the default action type

- Better preloaded state types

- Internal subscription improvements

- Still marks `createStore` as deprecated!

- https://github.com/reduxjs/redux/releases/tag/v5.0.0

## React-Redux 9.0:

- *Now requires React 18 and RTK 2.0 / Redux 5.0*

Oh hey, that's my post!

(yes I spend too much time refreshing HN :) )

FWIW I did end up with a packaging combination that seems to work sufficiently. I never did fix the "FalseCJS" issue that `are-the-types-wrong` is detecting. I played with double-emitting TS typedefs, and the `tsup` tool _does_ actually have support for that now (added by Andrew Branch from the TS team). So it might be more feasible now. But ultimately I decided I was tired of messing with packaging setup and that what I've got is good enough. (hopefully)

We're actually about to launch Redux Toolkit 2.0 and Redux 5.0 this week, assuming the last couple pieces come together. Here's the latest RCs - you can see the current `package.json` files in there:

- https://github.com/reduxjs/redux-toolkit/releases/tag/v2.0.0...

- https://github.com/reduxjs/redux/releases/tag/v5.0.0-rc.1

However, Redux, or pure Redux to be specific, can be quite verbose and boilerplate-heavy. It requires a significantly lengthy setup, which is where Redux Toolkit comes in handy, offering a simplified and more efficient way to set up and manage state in your React applications.

State manager is a tool or library that helps manage the state of an application. It provides a centralized store or container for storing and managing data that can be accessed and updated by different components in the application. A state manager solves several problems. Firstly, it is a good practice to separate data and the logic related to it from components. Secondly, when using local state and passing it between components, the code can become convoluted due to the potential for deep nesting of components. By having a global store, we can access and modify data from any component. Alongside React Context, Redux or MobX are commonly used as state management libraries. Learn more Learn more