ubelt
transformers
ubelt | transformers | |
---|---|---|
7 | 181 | |
712 | 126,516 | |
- | 2.6% | |
8.3 | 10.0 | |
7 days ago | about 12 hours ago | |
Python | Python | |
Apache License 2.0 | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
ubelt
-
Python libs that I wish were part of the standard library
I can't give you a stdlib, but I can give you a package with a lot of the basic functionality but still small enough that it installs quickly and has negligable overhead. The ubelt library is a set of 100ish utility functions and classes. It's similar to boltons, but I suppose it reflects a different perspective on what's useful.
-
How do you feel about vendored packages?
Number 3 is the one I feel most conflicted about. Specifically, I tout my ubelt library as having 0 required dependencies. However, it vendors two libraries: progiter and orderedset. The first of which I also maintain and the second of which I don't maintain, but have contributed to. It feels odd to have a single dependency for a library that would otherwise have zero. But at the same time it feels odd to maintain that code myself. Also if I didn't vendor it, it would not be included in the documentation, so there is that. I've recently been thinking I should split ubelt up into many smaller packages and then use ubelt as a "hub" to include them all. However, that's a lot more work than just maintaining one (still quite small) package, and I think having everything broken up with incur a lot of overhead at pip install time, so I'm very conflicted on the whole subject.
- Useful helper libraries
-
Python projects with best practices on Github?
I'm fairly happy with my ubelt library.
-
[D] What is some cool python magic(s) that you've learned over the years?
The ubelt.util_platform module is a good example of including references to similar functionality.
-
[P] best-of-ml-python: A ranked list of awesome machine learning Python libraries
I also have a utility library ubelt with 552 stars and 6.9k downloads / month.
transformers
-
XLSTM: Extended Long Short-Term Memory
Fascinating work, very promising.
Can you summarise how the model in your paper differs from this one ?
https://github.com/huggingface/transformers/issues/27011
-
AI enthusiasm #9 - A multilingual chatbotπ£πΈ
transformers is a package by Hugging Face, that helps you interact with models on HF Hub (GitHub)
-
Maxtext: A simple, performant and scalable Jax LLM
Is t5x an encoder/decoder architecture?
Some more general options.
The Flax ecosystem
https://github.com/google/flax?tab=readme-ov-file
or dm-haiku
https://github.com/google-deepmind/dm-haiku
were some of the best developed communities in the Jax AI field
Perhaps the βtraxβ repo? https://github.com/google/trax
Some HF examples https://github.com/huggingface/transformers/tree/main/exampl...
Sadly it seems much of the work is proprietary these days, but one example could be Grok-1, if you customize the details. https://github.com/xai-org/grok-1/blob/main/run.py
-
Lossless Acceleration of LLM via Adaptive N-Gram Parallel Decoding
The HuggingFace transformers library already has support for a similar method called prompt lookup decoding that uses the existing context to generate an ngram model: https://github.com/huggingface/transformers/issues/27722
I don't think it would be that hard to switch it out for a pretrained ngram model.
-
AI enthusiasm #6 - Finetune any LLM you wantπ‘
Most of this tutorial is based on Hugging Face course about Transformers and on Niels Rogge's Transformers tutorials: make sure to check their work and give them a star on GitHub, if you please β€οΈ
-
Schedule-Free Learning β A New Way to Train
* Superconvergence + LR range finder + Fast AI's Ranger21 optimizer was the goto optimizer for CNNs, and worked fabulously well, but on transformers, the learning rate range finder sadi 1e-3 was the best, whilst 1e-5 was better. However, the 1 cycle learning rate stuck. https://github.com/huggingface/transformers/issues/16013
-
Gemma doesn't suck anymore β 8 bug fixes
Thanks! :) I'm pushing them into transformers, pytorch-gemma and collabing with the Gemma team to resolve all the issues :)
The RoPE fix should already be in transformers 4.38.2: https://github.com/huggingface/transformers/pull/29285
My main PR for transformers which fixes most of the issues (some still left): https://github.com/huggingface/transformers/pull/29402
- HuggingFace Transformers: Qwen2
- HuggingFace Transformers Release v4.36: Mixtral, Llava/BakLlava, SeamlessM4T v2
- HuggingFace: Support for the Mixtral Moe
What are some alternatives?
best-of-web-python - π A ranked list of awesome python libraries for web development. Updated weekly.
fairseq - Facebook AI Research Sequence-to-Sequence Toolkit written in Python.
best-of-python-dev - π A ranked list of awesome python developer tools and libraries. Updated weekly.
sentence-transformers - Multilingual Sentence & Image Embeddings with BERT
best-of-jupyter - π A ranked list of awesome Jupyter Notebook, Hub and Lab projects (extensions, kernels, tools). Updated weekly.
llama - Inference code for Llama models
best-of-python - π A ranked list of awesome Python open-source libraries and tools. Updated weekly.
transformer-pytorch - Transformer: PyTorch Implementation of "Attention Is All You Need"
fastcore - Python supercharged for the fastai library
text-generation-webui - A Gradio web UI for Large Language Models. Supports transformers, GPTQ, AWQ, EXL2, llama.cpp (GGUF), Llama models.
best-of-ml-python - π A ranked list of awesome machine learning Python libraries. Updated weekly.
huggingface_hub - The official Python client for the Huggingface Hub.