[D] What is a good way to maintain code readability and code quality while scaling up complexity in libraries like Hugging Face?

• llama

  184 53,053 8.1 Python

  Inference code for Llama models

  

In transformers, they tried really hard to have a single function or method to deal with both self and cross attention mechanisms, masking, positional and relative encodings, interpolation etc. While it allows a user to use the same function/method for any model, it has led to severe parameter bloat. Just compare the original implementation of llama by FAIR with the implementation by HF to get an idea.

• transformers

  175 125,021 10.0 Python

  🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.

  

In transformers, they tried really hard to have a single function or method to deal with both self and cross attention mechanisms, masking, positional and relative encodings, interpolation etc. While it allows a user to use the same function/method for any model, it has led to severe parameter bloat. Just compare the original implementation of llama by FAIR with the implementation by HF to get an idea.

• WorkOS

  workos.com sponsored

  The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

  

• tinygrad

  17 23,864 10.0 Python

  You like pytorch? You like micrograd? You love tinygrad! ❤️

  

what do you think about tinygrad? I think its a good example of growing and well written, (partially) well documented library with many close to reference implementations

Suggest a related project