[D] What is a good way to maintain code readability and code quality while scaling up complexity in libraries like Hugging Face?

• llama

  184 53,908 8.0 Python

  Inference code for Llama models

  

In transformers, they tried really hard to have a single function or method to deal with both self and cross attention mechanisms, masking, positional and relative encodings, interpolation etc. While it allows a user to use the same function/method for any model, it has led to severe parameter bloat. Just compare the original implementation of llama by FAIR with the implementation by HF to get an idea.

• Scout Monitoring

  www.scoutapm.com featured

  Free Django app performance insights with Scout Monitoring. Get Scout setup in minutes, and let us sweat the small stuff. A couple lines in settings.py is all you need to start monitoring your apps. Sign up for our free tier today.

  

• transformers

  181 127,531 10.0 Python

  🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.

  

In transformers, they tried really hard to have a single function or method to deal with both self and cross attention mechanisms, masking, positional and relative encodings, interpolation etc. While it allows a user to use the same function/method for any model, it has led to severe parameter bloat. Just compare the original implementation of llama by FAIR with the implementation by HF to get an idea.

• tinygrad

  25 24,614 10.0 Python

  You like pytorch? You like micrograd? You love tinygrad! ❤️

  

what do you think about tinygrad? I think its a good example of growing and well written, (partially) well documented library with many close to reference implementations

Suggest a related project