Show HN: Finetune LLaMA-7B on commodity GPUs using your own text

This page summarizes the projects mentioned and recommended in the original post on news.ycombinator.com

Our great sponsors
  • WorkOS - The modern identity platform for B2B SaaS
  • InfluxDB - Power Real-Time Data Analytics at Scale
  • SaaSHub - Software Alternatives and Reviews
  • minimal-llama

    I've been playing around with https://github.com/zphang/minimal-llama/ and https://github.com/tloen/alpaca-lora/blob/main/finetune.py, and wanted to create a simple UI where you can just paste text, tweak the parameters, and finetune the model quickly using a modern GPU.

    To prepare the data, simply separate your text with two blank lines.

    There's an inference tab, so you can test how the tuned model behaves.

    This is my first foray into the world of LLM finetuning, Python, Torch, Transformers, LoRA, PEFT, and Gradio.

    Enjoy!

  • simple-llm-finetuner

    Discontinued Simple UI for LLM Model Finetuning

  • WorkOS

    The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

  • alpaca-lora

    Instruct-tune LLaMA on consumer hardware

    I've been playing around with https://github.com/zphang/minimal-llama/ and https://github.com/tloen/alpaca-lora/blob/main/finetune.py, and wanted to create a simple UI where you can just paste text, tweak the parameters, and finetune the model quickly using a modern GPU.

    To prepare the data, simply separate your text with two blank lines.

    There's an inference tab, so you can test how the tuned model behaves.

    This is my first foray into the world of LLM finetuning, Python, Torch, Transformers, LoRA, PEFT, and Gradio.

    Enjoy!

  • LLM-As-Chatbot

    LLM as a Chatbot Service

  • dalai

    The simplest way to run LLaMA on your local machine

  • FlexGen

    Running large language models on a single GPU for throughput-oriented scenarios.

    > With no real knowledge of LLM and only recently started to understand what LLM terms mean, such as 'model, inference, LLM model, intruction set, fine tuning' whatelse do you think is required to make a took like yours?

    This was mee a few weeks ago. I got interested in all this when FlexGen (https://github.com/FMInference/FlexGen) was announced, which allowed to run inference using OPT model on consumer hardware. I'm an avid user of Stable Diffusion, and I wanted to see if I can have an SD equivalent of ChatGPT.

    Not understanding the details of hyperparameters or terminology, I basically asked ChatGPT to explain to me what these things are:

       Explain to someone who is a software engineer with limited knowledge of ML terms or linear algebra, what is "feed forward" and "self-attention" in the context of ML and large language models. Provide examples when possible.

  • text-generation-webui

    A Gradio web UI for Large Language Models. Supports transformers, GPTQ, AWQ, EXL2, llama.cpp (GGUF), Llama models.

  • InfluxDB

    Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  • OpenChatKit

  • peft

    🤗 PEFT: State-of-the-art Parameter-Efficient Fine-Tuning.

  • mesh-transformer-jax

    Model parallel transformers in JAX and Haiku

  • WebChatRWKVstic

    Discontinued ChatGPT-like Web UI for RWKVstic

  • whisper.cpp

    Port of OpenAI's Whisper model in C/C++

  • GPTQ-for-LLaMa

    4 bits quantization of LLaMA using GPTQ

  • petals

    🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading

  • SaaSHub

    SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts