Mixtures of Experts

• t5x

  7 2,503 8.5 Python
  

Google have released the models and code for the Switch Transformer from Fedus et al. (2021) under the Apache 2.0 licence. [0]

There's also OpenMoE - an open-source effort to train a mixture of experts model. Currently they've released a model with 8 billion parameters. [1]

[0] https://github.com/google-research/t5x/blob/main/docs/models...

[1] https://github.com/XueFuzhao/OpenMoE

• OpenMoE

  8 1,206 8.6 Python

  A family of open-sourced Mixture-of-Experts (MoE) Large Language Models

Google have released the models and code for the Switch Transformer from Fedus et al. (2021) under the Apache 2.0 licence. [0]

There's also OpenMoE - an open-source effort to train a mixture of experts model. Currently they've released a model with 8 billion parameters. [1]

[0] https://github.com/google-research/t5x/blob/main/docs/models...

[1] https://github.com/XueFuzhao/OpenMoE

• InfluxDB

  www.influxdata.com featured

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

Suggest a related project