TencentPretrain
simpleT5
| TencentPretrain | simpleT5 | |
|---|---|---|
| 1 | 2 | |
| 983 | 380 | |
| 1.1% | - | |
| 7.6 | 2.5 | |
| 9 days ago | 12 months ago | |
| Python | Python | |
| GNU General Public License v3.0 or later | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
TencentPretrain
simpleT5
-
Transformers: How to compare performance to base model?
Currently I just took ~42000 samples and trained a translation task directly on codeT5 with https://github.com/Shivanandroy/simpleT5. Validation loss and at least the qualitative results are not to bad. Im now going to try to compare it to the base codeT5-model with the *.loss function as suggested above.
-
[P] SimpleT5 : Train T5 models in just 3 lines of code
🌟GitHub: https://github.com/Shivanandroy/simpleT5 🌟Medium: https://snrspeaks.medium.com/simplet5-train-t5-models-in-just-3-lines-of-code-by-shivanand-roy-2021-354df5ae46ba 🌟Colab Notebook: https://colab.research.google.com/drive/1JZ8v9L0w0Ai3WbibTeuvYlytn0uHMP6O?usp=sharing
What are some alternatives?
tiger - Open Source LLM toolkit to build trustworthy LLM applications. TigerArmor (AI safety), TigerRAG (embedding, RAG), TigerTune (fine-tuning)
reformer-pytorch - Reformer, the efficient Transformer, in Pytorch
alpaca-lora - Instruct-tune LLaMA on consumer hardware
datatap-python - Focus on Algorithm Design, Not on Data Wrangling
LLM-Adapters - Code for our EMNLP 2023 Paper: "LLM-Adapters: An Adapter Family for Parameter-Efficient Fine-Tuning of Large Language Models"
ModelZoo.pytorch - Hands on Imagenet training. Unofficial ModelZoo project on Pytorch. MobileNetV3 Top1 75.64🌟 GhostNet1.3x 75.78🌟
llama-classification - Text classification with Foundation Language Model LLaMA
frame-semantic-transformer - Frame Semantic Parser based on T5 and FrameNet
stanford_alpaca - Code and documentation to train Stanford's Alpaca models, and generate the data.
KeyPhraseTransformer - KeyPhraseTransformer lets you quickly extract key phrases, topics, themes from your text data with T5 transformer | Keyphrase extraction | Keyword extraction
awesome-pretrained-chinese-nlp-models - Awesome Pretrained Chinese NLP Models，高质量中文预训练模型&大模型&多模态模型&大语言模型集合
fastT5 - ⚡ boost inference speed of T5 models by 5x & reduce the model size by 3x.