mixture-of-experts
pytorch-tutorial
| mixture-of-experts | pytorch-tutorial | |
|---|---|---|
| 2 | 3 | |
| 835 | 29,128 | |
| - | - | |
| 5.3 | 0.0 | |
| 16 days ago | 9 months ago | |
| Python | Python | |
| GNU General Public License v3.0 only | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
mixture-of-experts
- [Rumor] Potential GPT-4 architecture description
-
Local and Global loss
I have a requirement of training pipeline similar to Mixture of Experts (https://github.com/davidmrau/mixture-of-experts/blob/master/moe.py) but I want to train the Experts on a local loss for 1 epoch before predicting outputs from them (which would then be concatenated for the global loss of MoE). Can anyone suggest what’s the best way to set up this training pipeline?
pytorch-tutorial
-
PyTorch - What does contiguous() do?
I was going through this example of a LSTM language model on github (link).What it does in general is pretty clear to me. But I'm still struggling to understand what calling contiguous() does, which occurs several times in the code.
-
How to 'practice' pytorch after finishing its basic tutorial?
I tried to move straight to practicing implementing papers and trying to understand other people's codes but failed miserably. I feel like there was too much of a gap between the basic tutorial and being able to implement ideas into code....hence the question: Is there any resource/way to practice pytorch in general? I did find this and this, but I just wanted to hear what others have gone through to become better at PyTorch up to the point they can build stuff from their own ideas
- [P] Probabilistic Machine Learning: An Introduction, Kevin Murphy's 2021 e-textbook is out
What are some alternatives?
transformers - 🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.
InceptionTime - InceptionTime: Finding AlexNet for Time Series Classification
mmdetection - OpenMMLab Detection Toolbox and Benchmark
Conv-TasNet - A PyTorch implementation of Conv-TasNet described in "TasNet: Surpassing Ideal Time-Frequency Masking for Speech Separation" with Permutation Invariant Training (PIT).
hivemind - Decentralized deep learning in PyTorch. Built to train models on thousands of volunteers across the world.
pytorch-grad-cam - Advanced AI Explainability for computer vision. Support for CNNs, Vision Transformers, Classification, Object detection, Segmentation, Image similarity and more.
yolov5 - YOLOv5 🚀 in PyTorch > ONNX > CoreML > TFLite
BigGAN-PyTorch - The author's officially unofficial PyTorch BigGAN implementation.
Real-Time-Voice-Cloning - Clone a voice in 5 seconds to generate arbitrary speech in real-time
bonito - A PyTorch Basecaller for Oxford Nanopore Reads
tutel - Tutel MoE: An Optimized Mixture-of-Experts Implementation
OpenNMT-py - Open Source Neural Machine Translation and (Large) Language Models in PyTorch