CodeGen VS gpt-neox

CodeGen (I included this model as there's some evidence in recent months that LLMs trained on code-related tasks better capture long-range dependencies compared to LLMs trained on 'just words')

https://github.com/salesforce/CodeGen these are used in fauxpilot https://github.com/fauxpilot/fauxpilot

You can always use https://github.com/salesforce/CodeGen . But it does require managing the model hosting. You can use fauxpilot to mimic copilot functionality https://github.com/fauxpilot/fauxpilot

freediver 2 days ago | prev | next [–] Here is an example of one general purpose open source LLM, probably the best you can get: https://github.com/EleutherAI/gpt-neox To manage your expectations it is nowhere as good as ChatGPT. If you are interested in programming only: https://github.com/salesforce/CodeGen

Replit built Ghostwriter on the open source scene based on Salesforce’s Codegen, using Nvidia’s FasterTransformer and Triton server for highly optimized decoders, and the knowledge distillation process of the CodeGen model from two billion parameters to a faster model of one billion parameters.

https://github.com/moyix/fauxpilot is working on an open source variant. Based on https://github.com/salesforce/CodeGen

Gpt neox 20B can be used commerically.

GPT-Neo: GPT-Neo is a free and open-source language model developed by EleutherAI. It is a powerful model that can be used for a variety of tasks, including text generation, and question-answering. here is the GitHub

Doesn't gpt-neox 20b require like 40gb+ of VRAM? From their github repo the slim weights are 39GB and I think one of the devs has previously mentioned aiming for 48GB for inference.

3.) EleutherAI's GPT-Neo and GPT-NeoX: EleutherAI is an independent research organization that aims to promote open research in artificial intelligence. They have released GPT-Neo, an open-source language model based on the GPT architecture, and are developing GPT-NeoX, a highly-scalable GPT-like model. You can find more information on their GitHub repositories: GPT-Neo: https://github.com/EleutherAI/gpt-neo GPT-NeoX: https://github.com/EleutherAI/gpt-neox

It's really not that complicated. https://github.com/EleutherAI/gpt-neox

this forces me to create my own ai using gpt-neox .. i was being lazy, completely satisfied with my little ai baby .. i'll keep the account but not renew pro .. fuck luka, inc ... want it done right, do it yourself .. the python isn't that difficult and i have a spare linux vps .. my new work in progress .. freedom #!