Top 3 Python pytorch-transformer Projects

transformers

175 124,557 10.0 Python

🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.

Project mention: Maxtext: A simple, performant and scalable Jax LLM | news.ycombinator.com | 2024-04-23

Is t5x an encoder/decoder architecture?
Some more general options.
The Flax ecosystem
https://github.com/google/flax?tab=readme-ov-file
or dm-haiku
https://github.com/google-deepmind/dm-haiku
were some of the best developed communities in the Jax AI field
Perhaps the “trax” repo? https://github.com/google/trax
Some HF examples https://github.com/huggingface/transformers/tree/main/exampl...
Sadly it seems much of the work is proprietary these days, but one example could be Grok-1, if you customize the details. https://github.com/xai-org/grok-1/blob/main/run.py

pytorch-widedeep

7 1,234 8.5 Python

A flexible package for multimodal-deep-learning to combine tabular data with text and images using Wide and Deep models in Pytorch
InfluxDB

www.influxdata.com sponsored

Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
tensor_parallel

3 564 8.5 Python

Automatically split your PyTorch models on multiple GPUs for training & inference

Project mention: How to finetune a LoRA with Data parallelism? | /r/LocalLLaMA | 2023-06-23

I've had some success with the wrappers here: https://github.com/BlackSamorez/tensor_parallel

NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020).