[D] PyTorch 2.0 Native Flash Attention 32k Context Window

• Pytorch

  336 77,783 10.0 Python

  Tensors and Dynamic neural networks in Python with strong GPU acceleration

  

You might look into https://github.com/pytorch/pytorch/pull/95793.

• ChatGPTCLIBot

  12 341 8.0 C++

  ChatGPT Bot in CLI with long term memory support using Embeddings.

• WorkOS

  workos.com sponsored

  The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

  

• longformer

  2 1,941 10.0 Python

  Longformer: The Long-Document Transformer

  

This approach has been shown to work. Longformer even provided a script that did this for you: https://github.com/allenai/longformer/blob/master/scripts/convert_model_to_long.ipynb

Suggest a related project