Our great sponsors
-
PaLM-rlhf-pytorch
Implementation of RLHF (Reinforcement Learning with Human Feedback) on top of the PaLM architecture. Basically ChatGPT but with PaLM
-
-
WorkOS
The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
-
-
petals
🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading
Possibly the "PaLM + RLHF - Pytorch" project:
https://github.com/lucidrains/PaLM-rlhf-pytorch
I don't much follow AI news beyond what I randomly happen to see on HN, but this might still be the largest open source model: https://github.com/yandex/YaLM-100B . There's discussion of it here: https://old.reddit.com/r/MachineLearning/comments/vpn0r1/d_h... - at the bottom of that page is a comment from someone who actually ran it in the cloud.
It's not possible post GPT2 for the reasons given by others.
Open communities with potential for involving yourself include Hugging Face and EleutherAI, the former perhaps more accessible, the latter an active Discord.
It's been a while since I spent time looking at them, I'm not sure if there is something you can easily get up and running with.
https://huggingface.co/
https://www.eleuther.ai/
https://github.com/bigscience-workshop/petals
Since my other account is shadow banned for some unexplained reason, I just wanted to mention the petal project. It's an attempt to bittorrent style distribute the load of running these large models. Good luck!