| SLIDE | goslide | |
|---|---|---|
| 3 | 1 | |
| 475 | 39 | |
| -0.4% | - | |
| 0.0 | 0.0 | |
| over 2 years ago | about 4 years ago | |
| Go | ||
| - | BSD 2-clause "Simplified" License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
SLIDE
-
Yandex opensources 100B parameter GPT-like model
That's pretty much what SLIDE [0] does. The driver was achieving performance parity with GPUs for CPU training, but presumably the same could apply to running inference on models too large to load into consumer GPU memory.
https://github.com/RUSH-LAB/SLIDE
- [R] CPU algorithm trains deep neural nets up to 15 times faster than top GPU trainers
- CPU-based algorithm trains deep neural nets up to 15 times faster than top GPU
goslide
What are some alternatives?
YaLM-100B - Pretrained language model with 100B parameters
Gorgonia - Gorgonia is a library that helps facilitate machine learning in Go.
lc0 - The rewritten engine, originally for tensorflow. Now all other backends have been ported here.
gpt-neox - An implementation of model parallel autoregressive transformers on GPUs, based on the Megatron and DeepSpeed libraries
olivia - 💁♀️Your new best friend powered by an artificial neural network
HashingDeepLearning - Codebase for "SLIDE : In Defense of Smart Algorithms over Hardware Acceleration for Large-Scale Deep Learning Systems"
Stockfish - A free and strong UCI chess engine
mesh-transformer-jax - Model parallel transformers in JAX and Haiku
YaLM-100B - Pretrained language model with 100B parameters