airoboros VS llama.cpp

Just wanted to drop a note that I overhauled the airoboros tool not the models to have most of the prompts I've been using to build the datasets, plus a couple extras.

airoboros: using large language models to fine-tune large language models (https://github.com/jondurbin/airoboros)

This is a fine-tuned LlaMa model, using completely synthetic training data created by https://github.com/jondurbin/airoboros

I used airoboros, a python tool I wrote, to generate the synthetic instruction response pairs, and included a jailbreak prompt to attempt to bypass OpenAI censorship. This is the only dataset used to fine-tune the model.

This is a 7b parameter, fine-tuned on 100k synthetic instruction/response pairs generated by gpt-3.5-turbo using my version of self-instruct airoboros

GitHub Repo

For anyone interested in exploring this, llama.cpp has an example implementation here:

https://github.com/ggerganov/llama.cpp/tree/master/examples/...

Getting started with LLMs can be intimidating. In this tutorial we will show you how to fine-tune a large language model using LoRA, facilitated by tools like llama.cpp and KitOps.

well https://github.com/ggerganov/llama.cpp/issues/6849

Have just done this recently for local chat with pdf feature in https://recurse.chat. (It's a macOS app that has built-in llama.cpp server and local vector database)

Running an embedding server locally is pretty straightforward:

- Get llama.cpp release binary: https://github.com/ggerganov/llama.cpp/releases