finetune-gpt2xl
quickai
Our great sponsors
| finetune-gpt2xl | quickai | |
|---|---|---|
| 9 | 7 | |
| 421 | 162 | |
| - | - | |
| 0.0 | 3.7 | |
| 11 months ago | 29 days ago | |
| Python | Python | |
| MIT License | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
finetune-gpt2xl
-
Fine-tuning?
git clone the finetuning repo https://github.com/Xirider/finetune-gpt2xl go into the finetuning repo, install the rest of the requirements, pip install -r requirements.txt
- Training text-generating models locally
-
Dataset For GPT Fine-Tuning
I would like to understand a little better how to organize texts for Fine-Tuning, especially for GPT Neo. I plan to use this repo procedure, where is the following notice,
-
How to share the finetuned model
In the code suggested in the video (and in the repo) the flag --fp16 is used. But reading the "DeepSpeed Integration" article it is said that,
- [D] I made a script that does all the work to deploy GPT-NEO on Windows 10. (Please Test)
-
[Project] Estimating fine-tuning cost
Finetuning GPT-NEO 2.7B on Wikitext (180mb) took me about 45 minutes on one preemptible V100 instance on google cloud. It cost 1.30$ per hour and therefore around 1 $. Here are the steps: https://github.com/Xirider/finetune-gpt2xl
-
[P] Guide: Finetune GPT2-XL (1.5 Billion Parameters, the biggest model) on a single 16 GB VRAM V100 Google Cloud instance with Huggingface Transformers using DeepSpeed
Here i explain the setup and commands to get it running: https://github.com/Xirider/finetune-gpt2xl
- Guide: Finetune GPT2-XL (1.5 Billion Parameters, the biggest model) on a single 16 GB VRAM V100 Google Cloud instance with Huggingface Transformers using DeepSpeed
quickai
- Show HN: QuickAI Version 2 Released
-
QuickAI version 2 released!
I originally released QuickAI here. I am very excited to announce version 2 of QuickAI
-
QuickAI is a Python library that makes it extremely easy to experiment with state-of-the-art Machine Learning models.
GitHub: https://github.com/geekjr/quickai
- Show HN: Quickai – Quickly experiment with state-of-the-art ML models
-
quickai - A Python library that makes it extremely easy to experiment with state-of-the-art Machine Learning models.
Yeah, totally agree. https://github.com/geekjr/quickai/blob/main/quickai/image_classification.py does really need some reworking. Dicts are the way to go. But once that's done, I think it could actually be a practical lib!
What are some alternatives?
detoxify - Trained models & code to predict toxic comments on all 3 Jigsaw Toxic Comment Challenges. Built using ⚡ Pytorch Lightning and 🤗 Transformers. For access to our API, please email us at [email protected].
Extracting-Training-Data-from-Large-Langauge-Models - A re-implementation of the "Extracting Training Data from Large Language Models" paper by Carlini et al., 2020
gpt-neo_dungeon - Colab notebooks to run a basic AI Dungeon clone using gpt-neo-2.7B
segyio - Fast Python library for SEGY files.
YOLOv6 - YOLOv6: a single-stage object detection framework dedicated to industrial applications.
chappie.ai - Generalized AI to perform a multitude of tasks written in python3
Note - Easily implement parallel training and distributed training. Machine learning library. Note.neuralnetwork.tf package include Llama2, Llama3, CLIP, ViT, ConvNeXt, SwiftFormer, etc, these models built with Note are compatible with TensorFlow and can be trained with TensorFlow.
happy-transformer - Happy Transformer makes it easy to fine-tune and perform inference with NLP Transformer models.
TabFormer - Code & Data for "Tabular Transformers for Modeling Multivariate Time Series" (ICASSP, 2021)
TensorLayer - Deep Learning and Reinforcement Learning Library for Scientists and Engineers
rtdl - Research on Tabular Deep Learning [Moved to: https://github.com/yandex-research/rtdl]