Ingest, store, & analyze all types of time series data in a fully-managed, purpose-built database. Keep data forever with low-cost storage and superior data compression. Learn more →
Top 23 Python Transformer Projects
-
vit-pytorch
Implementation of Vision Transformer, a simple way to achieve SOTA in vision classification with only a single transformer encoder, in Pytorch
Project mention: Is it easier to go from Pytorch to TF and Keras than the other way around? | /r/pytorch | 2023-05-13I also need to learn Pyspark so right now I am going to download the Fashion Mnist dataset, use Pyspark to downsize each image and put the into separate folders according to their labels (just to show employers I can do some basic ETL with Pyspark, not sure how I am going to load for training in Pytorch yet though). Then I am going to write the simplest Le Net to try to categorize the fashion MNIST dataset (results will most likely be bad but it's okay). Next, try to learn transfer learning in Pytorch for both CNN or maybe skip ahead to ViT. Ideally at this point I want to study the Attention mechanism a bit more and try to implement Simple Vit which I saw here: https://github.com/lucidrains/vit-pytorch/blob/main/vit_pytorch/simple_vit.py
-
PaddleNLP
👑 Easy-to-use and powerful NLP library with 🤗 Awesome model zoo, supporting wide-range of NLP tasks from research to industrial applications, including 🗂Text Classification, 🔍 Neural Search, ❓ Question Answering, ℹ️ Information Extraction, 📄 Document Intelligence, 💌 Sentiment Analysis and 🖼 Diffusion AIGC system etc.
-
Sonar
Write Clean Python Code. Always.. Sonar helps you commit clean code every time. With over 225 unique rules to find Python bugs, code smells & vulnerabilities, Sonar finds the issues while you focus on the work.
-
haystack
:mag: Haystack is an open source NLP framework to interact with your data using Transformer models and LLMs (GPT-4, ChatGPT and alike). Haystack offers production-ready tools to quickly build complex question answering, semantic search, text generation applications, and more.
-
Project mention: What are you building with LLMs? I'm writing an article about what people are building with LLMs | /r/programming | 2023-03-27
Hi everyone. I’m the creator of ChatLLaMA https://github.com/nebuly-ai/nebullvm/tree/main/apps/accelerate/chatllama, an opensource framework to train LLMs with limited resources and create There’s been amazing usage of LLMs in these days, from chatbots to retrieve about company’s product information, to cooking assistants for traditional dishes, and much more. And you? What you building or would love to build with LLMs? Let me know and I’ll share the article about your stories soon. https://qpvirevo4tz.typeform.com/to/T3PruEuE Cheers
-
RWKV-LM
RWKV is an RNN with transformer-level LLM performance. It can be directly trained like a GPT (parallelizable). So it's combining the best of RNN and transformer - great performance, fast inference, saves VRAM, fast training, "infinite" ctx_len, and free sentence embedding.
Project mention: New model: RWKV-4-Raven-7B-v12-Eng49%-Chn49%-Jpn1%-Other1%-20230530-ctx8192.pth | /r/LocalLLaMA | 2023-05-31See https://github.com/BlinkDL/RWKV-LM for details on the RWKV Language Model (100% RNN).
-
PaLM-rlhf-pytorch
Implementation of RLHF (Reinforcement Learning with Human Feedback) on top of the PaLM architecture. Basically ChatGPT but with PaLM
Project mention: How should I get an in-depth mathematical understanding of generative AI? | /r/datascience | 2023-05-18ChatGPT isn't open sourced so we don't know what the actual implementation is. I think you can read Open Assistant's source code for application design. If that is too much, try Open Chat Toolkit's source code for developer tools . If you need very bare implementation, you should go for lucidrains/PaLM-rlhf-pytorch.
-
Project mention: [D] [LoRA + weight merge every N step] for pre-training? | /r/MachineLearning | 2023-05-29
you could use a callback, like show here, https://github.com/huggingface/peft/issues/286 and call code to merge them here.
-
InfluxDB
Access the most powerful time series database as a service. Ingest, store, & analyze all types of time series data in a fully-managed, purpose-built database. Keep data forever with low-cost storage and superior data compression.
-
https://github.com/ggerganov/whisper.cpp https://speechbrain.github.io/
-
gpt-neox
An implementation of model parallel autoregressive transformers on GPUs, based on the DeepSpeed library.
Gpt neox 20B can be used commerically.
-
Project mention: Ask HN: Can someone ELI5 Transformers and the “Attention is all we need” paper | news.ycombinator.com | 2023-05-17
The Illustrated Transfomer ( https://jalammar.github.io/illustrated-transformer/ ) and Visualizing attention ( https://towardsdatascience.com/deconstructing-bert-part-2-vi... ), are both really good resources. For a more ELI5 approach this non-technical explainer ( https://www.parand.com/a-non-technical-explanation-of-chatgp... ) covers it at a high level.
-
DALLE-pytorch
Implementation / replication of DALL-E, OpenAI's Text to Image Transformer, in Pytorch
Here you go: https://github.com/lucidrains/DALLE-pytorch
-
deep-daze
Simple command line tool for text to image generation using OpenAI's CLIP and Siren (Implicit neural representation network). Technique was originally created by https://twitter.com/advadnoun
-
Try out this notebook which comes with the BERTopic repository.
-
simpletransformers
Transformers for Classification, NER, QA, Language Modelling, Language Generation, T5, Multi-Modal, and Conversational AI
-
yolov7_d2
🔥🔥🔥🔥 (Earlier YOLOv7 not official one) YOLO with Transformers and Instance Segmentation, with TensorRT acceleration! 🔥🔥🔥
Especially hilarious considering some other people ALSO jumped on the "we made an object detector so let's call it YOLOvX" wagon and released...
Something called YOLOv7.
-
x-transformers
A simple but complete full-attention transformer with a set of promising experimental features from various papers
Project mention: GPT-4 architecture: what we can deduce from research literature | news.ycombinator.com | 2023-03-14 -
courses
This repository is a curated collection of links to various courses and resources about Artificial Intelligence (AI) (by SkalskiP)
Project mention: GitHub - SkalskiP/courses: This repository is a curated collection of links to various courses and resources about Artificial Intelligence (AI) | /r/integratedai | 2023-05-26 -
musiclm-pytorch
Implementation of MusicLM, Google's new SOTA model for music generation using attention networks, in Pytorch
Has anyone tried to train this model : lucidrains/musiclm-pytorch: Implementation of MusicLM, Google's new SOTA model for music generation using attention networks, in Pytorch (github.com) ? Could you provide any useful resources that can help me? Or share your process?
-
Project mention: Vid2Seq: A pretrained visual language model for describing multi-event videos | news.ycombinator.com | 2023-03-17
Anyone figured out how to run this against a video?
https://github.com/google-research/scenic/tree/main/scenic/p... has an example showing how to "train Vid2Seq on YouCook2" using "python -m scenic.projects.vid2seq.main", but I couldn't see the recipe for using it against a video to return a description.
-
Consider a free software alternative to this translation service, such as Argos Translate or its web based frontend LibreTranslate.
-
-
scikit-llm
Seamlessly integrate powerful language models like ChatGPT into scikit-learn for enhanced text analysis tasks.
-
Project mention: [D] NLP question: does fine-tuning train input embedding? | /r/MachineLearning | 2022-08-07
Usually in computer vision resnets, people finetune only the last layers, but in NLP you tune the entire model. There are also plenty of instances where people try to not do this, such as in adapters, however.
-
SaaSHub
SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives
NOTE:
The open source projects on this list are ordered by number of github stars.
The number of mentions indicates repo mentiontions in the last 12 Months or
since we started tracking (Dec 2020).
The latest post mention was on 2023-06-02.
Python Transformers related posts
- Introducing Basaran: self-hosted open-source alternative to the OpenAI text completion API
- Training musiclm
- Aviary: Compare Open Source LLMs for cost, latency and quality
- [N] Aviary: Comparing Open Source LLMs for cost, latency and quality
- AutoGPTQ - An easy-to-use LLMs quantization package with user-friendly apis, based on GPTQ algorithm
- AutoGPTQ - An easy-to-use LLMs quantization package with user-friendly apis, based on GPTQ algorithm
- AutoGPTQ - An easy-to-use LLMs quantization package with user-friendly apis, based on GPTQ algorithm
-
A note from our sponsor - InfluxDB
www.influxdata.com | 4 Jun 2023
Index
What are some of the best open-source Transformer projects in Python? This list will help you:
| Project | Stars | |
|---|---|---|
| 1 | vit-pytorch | 14,271 |
| 2 | PaddleNLP | 9,259 |
| 3 | haystack | 8,976 |
| 4 | nebuly | 8,152 |
| 5 | RWKV-LM | 7,913 |
| 6 | PaLM-rlhf-pytorch | 6,911 |
| 7 | peft | 6,457 |
| 8 | speechbrain | 6,010 |
| 9 | gpt-neox | 5,538 |
| 10 | bertviz | 5,256 |
| 11 | DALLE-pytorch | 5,222 |
| 12 | deep-daze | 4,386 |
| 13 | BERTopic | 4,249 |
| 14 | simpletransformers | 3,681 |
| 15 | yolov7_d2 | 3,061 |
| 16 | x-transformers | 2,975 |
| 17 | courses | 2,640 |
| 18 | musiclm-pytorch | 2,441 |
| 19 | scenic | 2,208 |
| 20 | argos-translate | 1,957 |
| 21 | reformer-pytorch | 1,906 |
| 22 | scikit-llm | 1,678 |
| 23 | adapter-transformers | 1,633 |
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com