Mixture-of-experts Alternatives

Similar projects and alternatives to mixture-of-experts

openpilot

839 47,461 10.0 Python mixture-of-experts VS openpilot

openpilot is an open source driver assistance system. openpilot performs the functions of Automated Lane Centering and Adaptive Cruise Control for 250+ supported car makes and models.
pytorch-tutorial

3 29,093 0.0 Python mixture-of-experts VS pytorch-tutorial

PyTorch Tutorial for Deep Learning Researchers
InfluxDB

www.influxdata.com sponsored

Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
transformers

175 124,557 10.0 Python mixture-of-experts VS transformers

🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.
mmdetection

23 27,742 8.7 Python mixture-of-experts VS mmdetection

OpenMMLab Detection Toolbox and Benchmark
hivemind

40 1,833 5.9 Python mixture-of-experts VS hivemind

Decentralized deep learning in PyTorch. Built to train models on thousands of volunteers across the world.
yolov5

129 46,738 8.9 Python mixture-of-experts VS yolov5

YOLOv5 🚀 in PyTorch > ONNX > CoreML > TFLite
Real-Time-Voice-Cloning

96 50,738 0.0 Python mixture-of-experts VS Real-Time-Voice-Cloning

Clone a voice in 5 seconds to generate arbitrary speech in real-time
WorkOS

workos.com sponsored

The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
tutel

1 648 6.1 Python mixture-of-experts VS tutel

Tutel MoE: An Optimized Mixture-of-Experts Implementation
ModuleFormer

1 215 5.7 Python mixture-of-experts VS ModuleFormer

ModuleFormer is a MoE-based architecture that includes two different types of experts: stick-breaking attention heads and feedforward experts. We released a collection of ModuleFormer-based Language Models (MoLM) ranging in scale from 4 billion to 8 billion parameters.

NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a better mixture-of-experts alternative or higher similarity.

Suggest an alternative to mixture-of-experts

mixture-of-experts reviews and mentions

Posts with mentions or reviews of mixture-of-experts. We have used some of these posts to build our list of alternatives and similar projects. The last one was on 2023-06-20.

[Rumor] Potential GPT-4 architecture description
2 projects | /r/LocalLLaMA | 20 Jun 2023
Local and Global loss
1 project | /r/pytorch | 4 Mar 2021

I have a requirement of training pipeline similar to Mixture of Experts (https://github.com/davidmrau/mixture-of-experts/blob/master/moe.py) but I want to train the Experts on a local loss for 1 epoch before predicting outputs from them (which would then be concatenated for the global loss of MoE). Can anyone suggest what’s the best way to set up this training pipeline?