Our great sponsors
-
datasets
🤗 The largest hub of ready-to-use datasets for ML models with fast, easy-to-use and efficient data manipulation tools
-
WorkOS
The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
It might be doable to set this up on an AWS machine with a beefy GPU or two. I haven't tried it yet though.
Once you have a model trained in Huggingface Transformers you'd be able to convert it using this script:
https://github.com/moyix/fauxpilot/blob/main/converter/huggi...
And then pass that my_code.json as the dataset name.
Thank you for sharing the command for finetuning! Is it possible to share your ds_config.json? I tried to finetune the 2B model on A100 (40GB) using your command, but got a CUDA out of memory error. The ds_config I used was the one from huggingface (https://github.com/huggingface/transformers/blob/main/tests/...).