Top 5 Python mixture-of-expert Projects

DeepSpeed

27 25,088 9.6 Python

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

Project mention: Using --deepspeed requires lots of manual tweaking | reddit.com/r/Oobabooga | 2023-05-11

Filed a discussion item on the deepspeed project: https://github.com/microsoft/DeepSpeed/discussions/3531
hivemind

30 1,502 6.5 Python

Decentralized deep learning in PyTorch. Built to train models on thousands of volunteers across the world.

Project mention: Do you think that AI research will slow down to a halt because of regulation? | reddit.com/r/singularity | 2023-05-21

not if we rise to meet that challenge. here's a few tools that facilitate AI research in the face of an advanced persistent threat: Hivemind- a distributed Pytorch framework
ONLYOFFICE

www.onlyoffice.com
sponsored

ONLYOFFICE Docs — document collaboration in your environment. Powerful document editing and collaboration in your app or environment. Ultimate security, API and 30+ ready connectors, SaaS or on-premises
mixture-of-experts

0 442 2.7 Python

PyTorch Re-Implementation of "The Sparsely-Gated Mixture-of-Experts Layer" by Noam Shazeer et al. https://arxiv.org/abs/1701.06538
tutel

0 411 6.2 Python

Tutel MoE: An Optimized Mixture-of-Experts Implementation
mixture-of-experts

0 261 1.6 Python

A Pytorch implementation of Sparsely-Gated Mixture of Experts, for massively increasing the parameter count of language models (by lucidrains)
CodiumAI

codium.ai
sponsored

TestGPT | Generating meaningful tests for busy devs. Get non-trivial tests (and trivial, too!) suggested right inside your IDE, so you can code smart, create more value, and stay confident when you push.

NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020). The latest post mention was on 2023-05-21.