Python mixture-of-experts

Open-source Python projects categorized as mixture-of-experts

Top 10 Python mixture-of-expert Projects

mixture-of-experts
  1. DeepSpeed

    DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

    Project mention: All Data and AI Weekly #193 - June 9, 2025 | dev.to | 2025-06-09
  2. InfluxDB

    InfluxDB – Built for High-Performance Time Series Workloads. InfluxDB 3 OSS is now GA. Transform, enrich, and act on time series data directly in the database. Automate critical tasks and eliminate the need to move data externally. Download now.

    InfluxDB logo
  3. optillm

    Optimizing inference proxy for LLMs

    Project mention: Show HN: DeepThink Plugin – Bring Gemini 2.5's parallel reasoning to open models | news.ycombinator.com | 2025-06-18

    - Increases inference time but significantly improves answer quality

    Link: https://github.com/codelion/optillm/tree/main/optillm/plugin...

  4. mixtral-offloading

    Run Mixtral-8x7B models in Colab or consumer desktops

  5. hivemind

    Decentralized deep learning in PyTorch. Built to train models on thousands of volunteers across the world.

  6. MoE-LLaVA

    Mixture-of-Experts for Large Vision-Language Models

  7. mixture-of-experts

    PyTorch Re-Implementation of "The Sparsely-Gated Mixture-of-Experts Layer" by Noam Shazeer et al. https://arxiv.org/abs/1701.06538

  8. mixture-of-experts

    A Pytorch implementation of Sparsely-Gated Mixture of Experts, for massively increasing the parameter count of language models (by lucidrains)

  9. SaaSHub

    SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

    SaaSHub logo
  10. mergoo

    A library for easily merging multiple LLM experts, and efficiently train the merged LLM.

  11. st-moe-pytorch

    Implementation of ST-Moe, the latest incarnation of MoE after years of research at Brain, in Pytorch

  12. attention-models

    Simplified Implementation of SOTA Deep Learning Papers in Pytorch

NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020).

Python mixture-of-experts discussion

Log in or Post with

Python mixture-of-experts related posts

  • Hertz-dev, the first open-source base model for conversational audio

    7 projects | news.ycombinator.com | 3 Nov 2024
  • Would anyone be interested in contributing to some group projects?

    4 projects | /r/learnmachinelearning | 24 Aug 2023
  • [Rumor] Potential GPT-4 architecture description

    2 projects | /r/LocalLLaMA | 20 Jun 2023
  • Hive mind:Train deep learning models on thousands of volunteers across the world

    1 project | news.ycombinator.com | 20 Jun 2023
  • Could a model not be trained by a decentralized network? Like Seti @ home or kinda-sorta like bitcoin. Petals accomplishes this somewhat, but if raw computer power is the only barrier to open-source I'd be happy to try organizing decentalized computing efforts

    2 projects | /r/LocalLLaMA | 17 Jun 2023
  • Orca (built on llama13b) looks like the new sheriff in town

    2 projects | /r/LocalLLaMA | 6 Jun 2023
  • Do you think that AI research will slow down to a halt because of regulation?

    1 project | /r/singularity | 21 May 2023
  • A note from our sponsor - InfluxDB
    www.influxdata.com | 23 Jun 2025
    InfluxDB 3 OSS is now GA. Transform, enrich, and act on time series data directly in the database. Automate critical tasks and eliminate the need to move data externally. Download now. Learn more →

Index

What are some of the best open-source mixture-of-expert projects in Python? This list will help you:

# Project Stars
1 DeepSpeed 39,055
2 optillm 2,550
3 mixtral-offloading 2,303
4 hivemind 2,209
5 MoE-LLaVA 2,182
6 mixture-of-experts 1,089
7 mixture-of-experts 763
8 mergoo 481
9 st-moe-pytorch 339
10 attention-models 4

Sponsored
InfluxDB – Built for High-Performance Time Series Workloads
InfluxDB 3 OSS is now GA. Transform, enrich, and act on time series data directly in the database. Automate critical tasks and eliminate the need to move data externally. Download now.
www.influxdata.com

Did you know that Python is
the 2nd most popular programming language
based on number of references?