[P] A Simpler @PyTorch Annotated Implementation of EleutherAI's 20B Language Model GPT-NeoX.

• neox

  3 96 7.4 Python

  Discontinued Simple Annotated implementation of GPT-NeoX in PyTorch

  

Github: https://github.com/labmlai/neox

• transformers

  174 124,557 10.0 Python

  🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.

  

There is also a pull request on huggingface transformers for GPT-NeoX-20B if anyone is interested: https://github.com/huggingface/transformers/pull/16659. It has worked for me

• WorkOS

  workos.com sponsored

  The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

  

• minimal-gpt-neox-20b

  1 126 0.0 Python

(Disclaimer: I also wrote a minimal single-GPU implementation of GPT-NeoX-20B in pure PyTorch here: https://github.com/zphang/minimal-gpt-neox-20b)

Suggest a related project