Why is ChatGPT and other large language models not feasible to be used locally in consumer grade hardware while Stable Diffusion is?

• KoboldAI-Client

  185 3,344 6.3 Python

KoboldAI is a self-hosted GUI that lets you download different models and run them locally. Im not an expert on these models since ive mostly just used stronger online services.

• taming-transformers

  35 5,354 0.0 Jupyter Notebook

  Taming Transformers for High-Resolution Image Synthesis

  

See https://arxiv.org/abs/2012.09841 for prior work. SD authors swap out the Transformer and language modelling objective with a UNet diffusion objective. In general, the more inductive bias your model has, the more efficient it can be. ChatGPT runs purely on a Transformer architecture, which has far fewer priors than a CNN and requires far more parameters as a result. This may not be the case in the future.

• WorkOS

  workos.com sponsored

  The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

  

• Open-Assistant

  329 36,622 9.1 Python

  OpenAssistant is a chat-based assistant that understands tasks, can interact with third-party systems, and retrieve information dynamically to do so.

  

Suggest a related project