gpt-2

Open-source projects categorized as gpt-2
Edit details
Language: + Python + Jupyter Notebook + TypeScript + Shell + Rust + Swift + HTML + Kotlin
Topics: Gpt gpt-3 NLP Machine Learning Deep Learning
Power Real-Time Data Analytics at Scale
Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
www.influxdata.com
featured

Top 23 gpt-2 Open-Source Projects

  • RWKV-LM

    RWKV is an RNN with transformer-level LLM performance. It can be directly trained like a GPT (parallelizable). So it's combining the best of RNN and transformer - great performance, fast inference, saves VRAM, fast training, "infinite" ctx_len, and free sentence embedding.

    • Project mention: Do LLMs need a context window? | news.ycombinator.com | 2023-12-25

    https://github.com/BlinkDL/RWKV-LM#rwkv-discord-httpsdiscord... lists a number of implementations of various versions of RWKV.

    https://github.com/BlinkDL/RWKV-LM#rwkv-parallelizable-rnn-w... :

    > RWKV: Parallelizable RNN with Transformer-level LLM Performance (pronounced as "RwaKuv", from 4 major params: R W K V)

    > RWKV is an RNN with Transformer-level LLM performance, which can also be directly trained like a GPT transformer (parallelizable). And it's 100% attention-free. You only need the hidden state at position t to compute the state at position t+1. You can use the "GPT" mode to quickly compute the hidden state for the "RNN" mode.

    > So it's combining the best of RNN and transformer - great performance, fast inference, saves VRAM, fast training, "infinite" ctx_len, and free sentence embedding (using the final hidden state).

    > "Our latest version is RWKV-6,*

  • TabNine

    AI Code Completions

  • InfluxDB

    Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

    InfluxDB logo

  • LoRA

    Code for loralib, an implementation of "LoRA: Low-Rank Adaptation of Large Language Models"

    • Project mention: DECT NR+: A technical dive into non-cellular 5G | news.ycombinator.com | 2024-04-02

    This seems to be an order of magnitude better than LoRa (https://lora-alliance.org/ not https://arxiv.org/abs/2106.09685). LoRa doesn't have all the features this one does like OFDM, TDM, FDM, and HARQ. I didn't know there's spectrum dedicated for DECT use.

  • Transformers-Tutorials

    This repository contains demos I made with the Transformers library by HuggingFace.

    • Project mention: AI enthusiasm #6 - Finetune any LLM you want💡 | dev.to | 2024-04-16

    Most of this tutorial is based on Hugging Face course about Transformers and on Niels Rogge's Transformers tutorials: make sure to check their work and give them a star on GitHub, if you please ❤️

  • GPT2-Chinese

    Chinese version of GPT2 training code, using BERT tokenizer.

  • awesome-pretrained-chinese-nlp-models

    Awesome Pretrained Chinese NLP Models，高质量中文预训练模型&大模型&多模态模型&大语言模型集合

  • picoGPT

    An unnecessarily tiny implementation of GPT-2 in NumPy.

    • Project mention: Understanding Automatic Differentiation in 30 lines of Python | news.ycombinator.com | 2023-08-24

    In that case, you might also enjoy https://jaykmody.com/blog/gpt-from-scratch/

    (here's the raw code: https://github.com/jaymody/picoGPT/blob/main/gpt2.py)

  • SaaSHub

    SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

    SaaSHub logo

  • xTuring

    Build, customize and control you own LLMs. From data pre-processing to fine-tuning, xTuring provides an easy way to personalize open-source LLMs. Join our discord community: https://discord.gg/TgHXuSJEk6

    • Project mention: I'm developing an open-source AI tool called xTuring, enabling anyone to construct a Language Model with just 5 lines of code. I'd love to hear your thoughts! | /r/machinelearningnews | 2023-09-07

    Explore the project on GitHub here.

  • rust-bert

    Rust native ready-to-use NLP pipelines and transformer-based models (BERT, DistilBERT, GPT2,...)

    • Project mention: How to leverage the state-of-the-art NLP models in Rust | /r/infinilabs | 2023-06-07

    brew install libtorch brew link libtorch brew ls --verbose libtorch | grep dylib export LIBTORCH=$(brew --cellar pytorch)/$(brew info --json pytorch | jq -r '.[0].installed[0].version') export LD_LIBRARY_PATH=${LIBTORCH}/lib:$LD_LIBRARY_PATH git clone https://github.com/guillaume-be/rust-bert.git cd rust-bert ORT_STRATEGY=system cargo run --example sentence_embeddings

  • DialoGPT

    Large-scale pretraining for dialogue

  • Code-LMs

    Guide to using pre-trained large language models of source code

    • Project mention: PolyCoder LLM integration | /r/neovim | 2023-05-23

  • transfer-learning-conv-ai

    🦄 State-of-the-Art Conversational AI with Transfer Learning

  • tabnine-vscode

    Visual Studio Code client for Tabnine. https://marketplace.visualstudio.com/items?itemName=TabNine.tabnine-vscode

  • Discord-AI-Chatbot

    This Discord chatbot is incredibly versatile. Powered incredibly fast Groq API

    • Project mention: Discord bot for OpenAI API Key? | /r/ChatGPT | 2023-12-07

  • awesome-transformer-nlp

    A curated list of NLP resources focused on Transformer networks, attention mechanism, GPT, BERT, ChatGPT, LLMs, and transfer learning.

  • this-word-does-not-exist

    This Word Does Not Exist

    • Project mention: Ask HN: How do you name software? | news.ycombinator.com | 2024-02-10

  • TencentPretrain

    Tencent Pre-training framework in PyTorch & Pre-trained Model Zoo

  • LLMFarm

    llama and other large language models on iOS and MacOS offline using GGML library.

    • Project mention: LLMFarm – LLaMA and other large language models locally on iOS and macOS | news.ycombinator.com | 2023-12-14

  • gpt_jailbreak_status

    This is a repository that aims to provide updates on the status of jailbreaking the OpenAI GPT language model.

    • Project mention: Ask HN: What have you built with LLMs? | news.ycombinator.com | 2024-02-05

  • TextRL

    Implementation of ChatGPT RLHF (Reinforcement Learning with Human Feedback) on any generation model in huggingface's transformer (blommz-176B/bloom/gpt/bart/T5/MetaICL)

  • tabnine-intellij

    Jetbrains IDEs client for TabNine. Compatible with all IntelliJ-based IDEs. https://plugins.jetbrains.com/plugin/12798-tabnine

  • gpt2bot

    Your new Telegram buddy powered by transformers

  • gpt-tokenizer

    JavaScript BPE Tokenizer Encoder Decoder for OpenAI's GPT-2 / GPT-3 / GPT-4. Port of OpenAI's tiktoken with additional features.

    • Project mention: I wrote a tokenizer for LLaMA that runs inside the browser | /r/LocalLLaMA | 2023-06-13

    There are more differences between GPT2 tokenizer and LLaMA tokenizer than only the vocab and merge data. It would take me some time to do implement a GPT2 tokenizer, and there are already good alternatives for those, so it wouldn't make sense to put time into making another one. For example, this library contains a GPT2 tokenizer: https://github.com/niieani/gpt-tokenizer

  • SaaSHub

    SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

    SaaSHub logo
NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020).

gpt-2 related posts

Index

What are some of the best open-source gpt-2 projects? This list will help you:

Project Stars
1 RWKV-LM 11,704
2 TabNine 10,415
3 LoRA 9,172
4 Transformers-Tutorials 7,875
5 GPT2-Chinese 7,360
6 awesome-pretrained-chinese-nlp-models 4,250
7 picoGPT 3,081
8 xTuring 2,524
9 rust-bert 2,434
10 DialoGPT 2,315
11 Code-LMs 1,721
12 transfer-learning-conv-ai 1,714
13 tabnine-vscode 1,343
14 Discord-AI-Chatbot 1,276
15 awesome-transformer-nlp 1,036
16 this-word-does-not-exist 1,013
17 TencentPretrain 983
18 LLMFarm 917
19 gpt_jailbreak_status 883
20 TextRL 518
21 tabnine-intellij 502
22 gpt2bot 424
23 gpt-tokenizer 383

Sponsored
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com