gpt-2

Open-source projects categorized as gpt-2
Edit details
Language: + Python + Jupyter Notebook + TypeScript + Shell + Rust + HTML + Swift + Kotlin
Topics: Gpt gpt-3 NLP Machine Learning Deep Learning
Power Real-Time Data Analytics at Scale
Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
www.influxdata.com
featured

Top 23 gpt-2 Open-Source Projects

  • RWKV-LM

    RWKV is an RNN with transformer-level LLM performance. It can be directly trained like a GPT (parallelizable). So it's combining the best of RNN and transformer - great performance, fast inference, saves VRAM, fast training, "infinite" ctx_len, and free sentence embedding.

    • Project mention: Do LLMs need a context window? | news.ycombinator.com | 2023-12-25

    https://github.com/BlinkDL/RWKV-LM#rwkv-discord-httpsdiscord... lists a number of implementations of various versions of RWKV.

    https://github.com/BlinkDL/RWKV-LM#rwkv-parallelizable-rnn-w... :

    > RWKV: Parallelizable RNN with Transformer-level LLM Performance (pronounced as "RwaKuv", from 4 major params: R W K V)

    > RWKV is an RNN with Transformer-level LLM performance, which can also be directly trained like a GPT transformer (parallelizable). And it's 100% attention-free. You only need the hidden state at position t to compute the state at position t+1. You can use the "GPT" mode to quickly compute the hidden state for the "RNN" mode.

    > So it's combining the best of RNN and transformer - great performance, fast inference, saves VRAM, fast training, "infinite" ctx_len, and free sentence embedding (using the final hidden state).

    > "Our latest version is RWKV-6,*

  • TabNine

    AI Code Completions

  • InfluxDB

    Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

    InfluxDB logo

  • LoRA

    Code for loralib, an implementation of "LoRA: Low-Rank Adaptation of Large Language Models"

    • Project mention: DECT NR+: A technical dive into non-cellular 5G | news.ycombinator.com | 2024-04-02

    This seems to be an order of magnitude better than LoRa (https://lora-alliance.org/ not https://arxiv.org/abs/2106.09685). LoRa doesn't have all the features this one does like OFDM, TDM, FDM, and HARQ. I didn't know there's spectrum dedicated for DECT use.

  • Transformers-Tutorials

    This repository contains demos I made with the Transformers library by HuggingFace.

    • Project mention: AI enthusiasm #6 - Finetune any LLM you want💡 | dev.to | 2024-04-16

    Most of this tutorial is based on Hugging Face course about Transformers and on Niels Rogge's Transformers tutorials: make sure to check their work and give them a star on GitHub, if you please ❤️

  • GPT2-Chinese

    Chinese version of GPT2 training code, using BERT tokenizer.

  • awesome-pretrained-chinese-nlp-models

    Awesome Pretrained Chinese NLP Models，高质量中文预训练模型&大模型&多模态模型&大语言模型集合

  • picoGPT

    An unnecessarily tiny implementation of GPT-2 in NumPy.

    • Project mention: Understanding Automatic Differentiation in 30 lines of Python | news.ycombinator.com | 2023-08-24

    In that case, you might also enjoy https://jaykmody.com/blog/gpt-from-scratch/

    (here's the raw code: https://github.com/jaymody/picoGPT/blob/main/gpt2.py)

  • SaaSHub

    SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

    SaaSHub logo

  • xTuring

    Build, customize and control you own LLMs. From data pre-processing to fine-tuning, xTuring provides an easy way to personalize open-source LLMs. Join our discord community: https://discord.gg/TgHXuSJEk6

    • Project mention: I'm developing an open-source AI tool called xTuring, enabling anyone to construct a Language Model with just 5 lines of code. I'd love to hear your thoughts! | /r/machinelearningnews | 2023-09-07

    Explore the project on GitHub here.

  • rust-bert

    Rust native ready-to-use NLP pipelines and transformer-based models (BERT, DistilBERT, GPT2,...)

    • Project mention: How to leverage the state-of-the-art NLP models in Rust | /r/infinilabs | 2023-06-07

    brew install libtorch brew link libtorch brew ls --verbose libtorch | grep dylib export LIBTORCH=$(brew --cellar pytorch)/$(brew info --json pytorch | jq -r '.[0].installed[0].version') export LD_LIBRARY_PATH=${LIBTORCH}/lib:$LD_LIBRARY_PATH git clone https://github.com/guillaume-be/rust-bert.git cd rust-bert ORT_STRATEGY=system cargo run --example sentence_embeddings

  • DialoGPT

    Large-scale pretraining for dialogue

  • Code-LMs

    Guide to using pre-trained large language models of source code

    • Project mention: PolyCoder LLM integration | /r/neovim | 2023-05-23

  • transfer-learning-conv-ai

    🦄 State-of-the-Art Conversational AI with Transfer Learning

  • tabnine-vscode

    Visual Studio Code client for Tabnine. https://marketplace.visualstudio.com/items?itemName=TabNine.tabnine-vscode

  • Discord-AI-Chatbot

    This Discord chatbot is incredibly versatile. Powered incredibly fast Groq API

    • Project mention: Discord bot for OpenAI API Key? | /r/ChatGPT | 2023-12-07

  • awesome-transformer-nlp

    A curated list of NLP resources focused on Transformer networks, attention mechanism, GPT, BERT, ChatGPT, LLMs, and transfer learning.

  • this-word-does-not-exist

    This Word Does Not Exist

    • Project mention: Ask HN: How do you name software? | news.ycombinator.com | 2024-02-10

  • TencentPretrain

    Tencent Pre-training framework in PyTorch & Pre-trained Model Zoo

  • gpt_jailbreak_status

    This is a repository that aims to provide updates on the status of jailbreaking the OpenAI GPT language model.

    • Project mention: Ask HN: What have you built with LLMs? | news.ycombinator.com | 2024-02-05

  • LLMFarm

    llama and other large language models on iOS and MacOS offline using GGML library.

    • Project mention: LLMFarm – LLaMA and other large language models locally on iOS and macOS | news.ycombinator.com | 2023-12-14

  • TextRL

    Implementation of ChatGPT RLHF (Reinforcement Learning with Human Feedback) on any generation model in huggingface's transformer (blommz-176B/bloom/gpt/bart/T5/MetaICL)

  • tabnine-intellij

    Jetbrains IDEs client for TabNine. Compatible with all IntelliJ-based IDEs. https://plugins.jetbrains.com/plugin/12798-tabnine

  • gpt2bot

    Your new Telegram buddy powered by transformers

  • gpt-tokenizer

    JavaScript BPE Tokenizer Encoder Decoder for OpenAI's GPT-2 / GPT-3 / GPT-4. Port of OpenAI's tiktoken with additional features.

    • Project mention: I wrote a tokenizer for LLaMA that runs inside the browser | /r/LocalLLaMA | 2023-06-13

    There are more differences between GPT2 tokenizer and LLaMA tokenizer than only the vocab and merge data. It would take me some time to do implement a GPT2 tokenizer, and there are already good alternatives for those, so it wouldn't make sense to put time into making another one. For example, this library contains a GPT2 tokenizer: https://github.com/niieani/gpt-tokenizer

  • SaaSHub

    SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

    SaaSHub logo
NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020).

gpt-2 related posts

  • Open source – Unsupervised captioning getting closer to supervised captioning

    1 project | news.ycombinator.com | 20 Apr 2024

  • DECT NR+: A technical dive into non-cellular 5G

    1 project | news.ycombinator.com | 2 Apr 2024

  • Training LLMs Taking Too Much Time? Technique you need to know to train it faster

    1 project | dev.to | 3 Mar 2024

  • Ask HN: How do you name software?

    1 project | news.ycombinator.com | 10 Feb 2024

  • Do LLMs need a context window?

    1 project | news.ycombinator.com | 25 Dec 2023

  • ArtBot for Stable Diffusion

    3 projects | news.ycombinator.com | 17 Dec 2023

  • OpenAI employee: GPT-4.5 rumor was a hallucination

    1 project | news.ycombinator.com | 17 Dec 2023
  • A note from our sponsor - InfluxDB
    www.influxdata.com | 1 May 2024
    Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality. Learn more →

Index

What are some of the best open-source gpt-2 projects? This list will help you:

Project Stars
1 RWKV-LM 11,619
2 TabNine 10,387
3 LoRA 9,046
4 Transformers-Tutorials 7,510
5 GPT2-Chinese 7,348
6 awesome-pretrained-chinese-nlp-models 4,193
7 picoGPT 3,081
8 xTuring 2,523
9 rust-bert 2,418
10 DialoGPT 2,315
11 Code-LMs 1,716
12 transfer-learning-conv-ai 1,711
13 tabnine-vscode 1,335
14 Discord-AI-Chatbot 1,270
15 awesome-transformer-nlp 1,031
16 this-word-does-not-exist 1,009
17 TencentPretrain 981
18 gpt_jailbreak_status 879
19 LLMFarm 895
20 TextRL 519
21 tabnine-intellij 501
22 gpt2bot 424
23 gpt-tokenizer 379

Sponsored
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com