airoboros VS gptqlora

Just wanted to drop a note that I overhauled the airoboros tool not the models to have most of the prompts I've been using to build the datasets, plus a couple extras.

airoboros: using large language models to fine-tune large language models (https://github.com/jondurbin/airoboros)

This is a fine-tuned LlaMa model, using completely synthetic training data created by https://github.com/jondurbin/airoboros

I used airoboros, a python tool I wrote, to generate the synthetic instruction response pairs, and included a jailbreak prompt to attempt to bypass OpenAI censorship. This is the only dataset used to fine-tune the model.

This is a 7b parameter, fine-tuned on 100k synthetic instruction/response pairs generated by gpt-3.5-turbo using my version of self-instruct airoboros

GitHub Repo

GPTQLoRA: Efficient Finetuning of Quantized LLMs with GPTQ (https://github.com/qwopqwop200/gptqlora/tree/main)

The difference from QLoRA is that GPTQ is used instead of NF4 (Normal Float4) + DQ (Double Quantization) for model quantization. The advantage is that you can expect better performance because it provides better quantization than conventional bitsandbytes. The downside is that it is a one-shot quantization methodology, so it is more inconvenient than bitsandbytes, and unlike bitsandbytes, it is not universal. I'm still experimenting, but it seems to work. At least, I hope it can be more options for people using LoRA. https://github.com/qwopqwop200/gptqlora/tree/main