[P] Finetuning a commercially viable open source LLM (Flan-UL2) using Alpaca, Dolly15K and LoRA

• alpaca-lora

  107 18,167 3.6 Jupyter Notebook

  Instruct-tune LLaMA on consumer hardware

• ue5-llama-lora

  16 450 2.9 Python

  A proof-of-concept project that showcases the potential for using small, locally trainable LLMs to create next-generation documentation tools.

• WorkOS

  workos.com sponsored

  The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

  

• dolly

  41 10,783 7.2 Python

  Databricks’ Dolly, a large language model trained on the Databricks Machine Learning Platform

  

Hi /u/meowkittykitty510, I'm embarking on a similar project, solely for teaching myself about finetuning LLMs. I initially chose Dolly (Pythia model) with modifications to the Dolly 2.0 trainer plus additional training data scraped from the web about The Expanse just as a fun way to test.

• AlpacaDataCleaned

  14 1,394 7.6 Python

  Alpaca dataset from Stanford, cleaned and curated

Suggest a related project