nn
functorch
Our great sponsors
nn | functorch | |
---|---|---|
26 | 11 | |
47,002 | 1,369 | |
6.6% | 0.8% | |
7.7 | 0.6 | |
25 days ago | 5 days ago | |
Jupyter Notebook | Jupyter Notebook | |
MIT License | BSD 3-clause "New" or "Revised" License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
nn
- [D] Looking for open source projects to contribute
-
PyTorch 1.10
We did a bunch of popular research paper implementations in PyTorch with notes (annotations); might be helpful.
-
[P] Annotated deep learning paper implementations
Website with side-by-side notes rendered: nn.labml.ai
You can create a pull request or start a discussion in GitHub issues. Even voting suggesting papers to implement and voting for them will be helpful. Here's a recent pull request for example
- PonderNet: Annotated PyTorch Implementation
-
Annotated PyTorch implementations of various deep learning normalization layers
Author here. From next month onwards we will pick papers to implement based on votes on Github issues https://github.com/lab-ml/nn/issues
Feel free to open an issue if there's a paper that you like implemented.
Git link: https://github.com/lab-ml/nn
I think this is awesome and personally cloned the repo so i can browse through the docs later without adding 'yet another tab' to my browser.
functorch
-
What is the most efficient approach to ensemble a pytorch actor-critic model?
I would suggest checking https://pytorch.org/functorch/ and https://github.com/metaopt/torchopt for efficient inference and training with ensembles (e.g., t be independent actors in a multi-agent setting or multiple critics).
-
[P] Multidimensional array batch indexing for pytorch and numpy
There were some bugs still with advanced indexing in an older release of functorch, I believe they should be fixed now though: https://github.com/pytorch/functorch/pull/862
-
Jax vs. Julia (Vs PyTorch)
Tangentially related but there is an effort to get some of the features of JAX into PyTorch: https://pytorch.org/functorch/
-
[D] Current State of JAX vs Pytorch?
Fwiw, composable vmap and stuff like that have also been implemented in PyTorch now - see functorch :) https://github.com/pytorch/functorch
-
[D] Ideal deep learning library
Fwiw, itโs not like Pytorchโs design prevents function transformations from being implemented. See functorch for an example of grad/vmap function transforms: https://github.com/pytorch/functorch
-
[P] Made Some Pytorch Modules For Agent Systems
You may find vmap from functorch to be quite useful: https://github.com/pytorch/functorch
-
[D] Are you using PyTorch or TensorFlow going into 2022?
If you're interested in function transformations in PyTorch, try out functorch :) https://github.com/pytorch/functorch
- PyTorch: Where we are headed and why it looks a lot like Julia (but not exactly)
-
Show HN: How does Jax allocate memory on a TPU? An interactive C++ walkthrough
The pytorch programming model is just really hard to adapt to an XLA-like compiler. Imperative python code doesn't translate to an ML graph compiler particularly well; Jax's API is functional, so it's easier to translate to the XLA API. By contrast, torch/xla uses "lazy tensors" that record the computation graph and compile when needed. The trouble is, if the compute graph changes from run to run, you end up recompiling a lot.
I guess in Jax you'd just only apply `jax.jit` to the parts where the compute graph is static? I'd be curious to see examples of how this works in practice. Fwiw, there's an offshoot of pytorch that is aiming to provide this sort of API (see https://github.com/pytorch/functorch and look at eager_compilation.py).
(Disclaimer: I worked on this until quite recently.)
-
PyTorch 1.10
https://github.com/pytorch/functorch) but not the second.
Disclaimer: I work on PyTorch, and Functorch more specifically, although my opinions here aren't on behalf of PyTorch.
What are some alternatives?
GFPGAN-for-Video-SR - A colab notebook for video super resolution using GFPGAN
jax - Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more
labml - ๐ Monitor deep learning model training and hardware usage from your mobile phone ๐ฑ
onnx-simplifier - Simplify your onnx model
ZoeDepth - Metric depth estimation from a single image
Basic-UI-for-GPT-J-6B-with-low-vram - A repository to run gpt-j-6b on low vram machines (4.2 gb minimum vram for 2000 token context, 3.5 gb for 1000 token context). Model loading takes 12gb free ram.
Behavior-Sequence-Transformer-Pytorch - This is a pytorch implementation for the BST model from Alibaba https://arxiv.org/pdf/1905.06874.pdf
DFL-Colab - DeepFaceLab fork which provides IPython Notebook to use DFL with Google Colab
Siren-fastai2 - Unofficial implementation of 'Implicit Neural Representations with Periodic Activation Functions'
torch2trt - An easy to use PyTorch to TensorRT converter
BinaryBuilder.jl - Binary Dependency Builder for Julia