Finetuning on multiple GPUs

• bitsandbytes

  61 5,389 9.4 Python

  Accessible large language models via k-bit quantization for PyTorch.

If it also has QLoRA that would be the best but afaik it's not implemented in bitsandbytes yet?

• text-generation-webui-testing

  9 22 9.9 Python

  A fork of textgen that still supports V1 GPTQ, 4-bit lora and other GPTQ models besides llama.

i've never tried that particular one. everything else I threw at it trained through : https://github.com/Ph0rk0z/text-generation-webui-testing/ successfully.

• WorkOS

  workos.com sponsored

  The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

  

• GPTQ-Merged

  2 2 8.1 Python

  trying to make sense of it

Probably need to add universal support to the native functions because it uses llama only. If you edit the load_llama functions in autograd py to use generic stuff like this: https://github.com/Ph0rk0z/GPTQ-Merged/blob/dual-model/src/alpaca_lora_4bit/autograd_4bit.py it has a good chance of working. Might need to also add trust_remote_code.

• axolotl

  29 5,641 9.8 Python

  Go ahead and axolotl questions

  

Finetuning on multiple GPUs works pretty much out of the box for every finetune project I've tried. Here's the best finetune codebase I'd found that supports QLoRA: https://github.com/OpenAccess-AI-Collective/axolotl

Suggest a related project