An implementation of model parallel GPT-3-like models on GPUs, based on the DeepSpeed library. Designed to be able to train models in the hundreds of billions of parameters or larger. (by EleutherAI)


Basic gpt-neox repo stats
1 day ago

EleutherAI/gpt-neox is an open source project licensed under Apache License 2.0 which is an OSI approved license.

Gpt-neox Alternatives

Similar projects and alternatives to gpt-neox

  • GitHub repo gpt-neo

    An implementation of model parallel GPT-2 and GPT-3-style models using the mesh-tensorflow library.

  • GitHub repo kiri

    Backprop makes it simple to use, finetune, and deploy state-of-the-art ML models.

  • GitHub repo aid_adventure_vulnerability_report

    Report and source code detailing the AI Dungeon private adventure vulnerability

  • GitHub repo Pytorch

    Tensors and Dynamic neural networks in Python with strong GPU acceleration

  • GitHub repo fairseq

    Facebook AI Research Sequence-to-Sequence Toolkit written in Python.

  • GitHub repo DeepSpeed

    DeepSpeed is a deep learning optimization library that makes distributed training easy, efficient, and effective.

  • GitHub repo haystack

    :mag: End-to-end Python framework for building natural language search interfaces to data. Leverages Transformers and the State-of-the-Art of NLP. Supports DPR, Elasticsearch, Hugging Face’s Hub, and much more!

  • GitHub repo lm-evaluation-harness

    A framework for few-shot evaluation of autoregressive language models.

NOTE: The number of mentions on this list indicates mentions on common posts. Hence, a higher number means a better gpt-neox alternative or higher similarity.


Posts where gpt-neox has been mentioned. We have used some of these posts to build our list of alternatives and similar projects - the last one was on 2021-04-30.
  • 'OpenAI' is a misnomer and a company with way too much power.
    The cat is out of the bag, this tech is not going away, and will very shortly be a regular part of life. OpenAI's pretentious decision to drop their nonprofit status and start charging for a product they received money to develop and release for free is asinine.
  • Thank You! (Petition Update: Day 4) (Too long for
    AI Dungeon Scenario/Adventure Downloader (save your creations for use with a different AI): GPT-Neo (Google Colab): GPT-Neo (repository): GPT-NeoX (repository): New, censorship-free AI Dungeon subreddit:
  • So, is The Infinite Story a viable switch from AI Dungeon until they get their shit together again? | 2021-04-28
    And this is the GitHub of this AI, I guess.
  • Update to Our Community | 2021-04-28
    Godspeed to the people at
  • Fuck this drama, let's just create our own, open-source AID called "OpenDungeon". Anyone agree? | 2021-04-28
    Yeah, open-source, OpenAI would have laughed at the idea, but there's something here.
  • This AI Can Generate Convincing Text—and Anyone Can Use It
    'Cept it's under active development.
  • GPT Neo: open-source GPT-3-like model with pretrained weights available | 2021-03-21
    GPT-NeoX, which is a model from the same group but using GPUs instead of TPUs, uses techniques from DeepSpeed:

  • Zero-3 Offload: Scale DL models to trillion parameters without code changes | 2021-03-13
    GPT-NeoX is an example project that is using deepspeed and Zero-3 offloading. The wider project intend to train a GPT-3 sized model and release it freely to the world. | 2021-03-13
    Looks like they got it working recently | 2021-03-13
  • Tom Scott: I asked an AI for video ideas, and they were actually good | 2021-02-23
    They have another GPT project as well. Not sure what the difference between the two are as they are both being updated.