functorch
BinaryBuilder.jl
Our great sponsors
functorch | BinaryBuilder.jl | |
---|---|---|
11 | 5 | |
1,372 | 379 | |
1.0% | 1.6% | |
0.0 | 6.5 | |
1 day ago | 17 days ago | |
Jupyter Notebook | Julia | |
BSD 3-clause "New" or "Revised" License | GNU General Public License v3.0 or later |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
functorch
-
What is the most efficient approach to ensemble a pytorch actor-critic model?
I would suggest checking https://pytorch.org/functorch/ and https://github.com/metaopt/torchopt for efficient inference and training with ensembles (e.g., t be independent actors in a multi-agent setting or multiple critics).
-
[P] Multidimensional array batch indexing for pytorch and numpy
There were some bugs still with advanced indexing in an older release of functorch, I believe they should be fixed now though: https://github.com/pytorch/functorch/pull/862
- Functorch: Jax-like composable function transforms for PyTorch
-
Jax vs. Julia (Vs PyTorch)
Tangentially related but there is an effort to get some of the features of JAX into PyTorch: https://pytorch.org/functorch/
-
[D] Current State of JAX vs Pytorch?
Fwiw, composable vmap and stuff like that have also been implemented in PyTorch now - see functorch :) https://github.com/pytorch/functorch
-
[D] Ideal deep learning library
Fwiw, it’s not like Pytorch’s design prevents function transformations from being implemented. See functorch for an example of grad/vmap function transforms: https://github.com/pytorch/functorch
-
[P] Made Some Pytorch Modules For Agent Systems
You may find vmap from functorch to be quite useful: https://github.com/pytorch/functorch
-
[D] Are you using PyTorch or TensorFlow going into 2022?
If you're interested in function transformations in PyTorch, try out functorch :) https://github.com/pytorch/functorch
- PyTorch: Where we are headed and why it looks a lot like Julia (but not exactly)
-
Show HN: How does Jax allocate memory on a TPU? An interactive C++ walkthrough
The pytorch programming model is just really hard to adapt to an XLA-like compiler. Imperative python code doesn't translate to an ML graph compiler particularly well; Jax's API is functional, so it's easier to translate to the XLA API. By contrast, torch/xla uses "lazy tensors" that record the computation graph and compile when needed. The trouble is, if the compute graph changes from run to run, you end up recompiling a lot.
I guess in Jax you'd just only apply `jax.jit` to the parts where the compute graph is static? I'd be curious to see examples of how this works in practice. Fwiw, there's an offshoot of pytorch that is aiming to provide this sort of API (see https://github.com/pytorch/functorch and look at eager_compilation.py).
(Disclaimer: I worked on this until quite recently.)
BinaryBuilder.jl
-
Is Julia suitable today as a scripting language?
There are some efforts and the startup times are getting better with every release and there's BinaryBuilder.jl.
-
Because cross-compiling binaries for Windows is easier than building natively
There is the Julia package https://github.com/JuliaPackaging/BinaryBuilder.jl which creates an environment that fakes being another, but with the correct compilers and SDKs . It's used to build all the binary dependencies
-
Discussion Thread
https://binarybuilder.org/. You can do it manually obviously, but this is easier.
-
PyTorch: Where we are headed and why it looks a lot like Julia (but not exactly)
> The main pain point is probably the lack of standard, multi-environment packaging solutions for natively compiled code.
Are you talking about something like BinaryBuilder.jl[1], which provides native binaries as julia-callable wrappers?
--
[1] https://binarybuilder.org
-
What to do about GPU packages on PyPI?
Julia did that for binary dependencies for a few years, with adapters for several linux platforms, homebrew, and for cross-compiled RPMs for Windows. It worked, to a degree -- less well on Windows -- but the combinatorial complexity led to many hiccups and significant maintenance effort. Each Julia package had to account for the peculiarities of each dependency across a range of dependency versions and packaging practices (linkage policies, bundling policies, naming variations, distro versions) -- and this is easier in Julia than in (C)Python because shared libraries are accessed via locally-JIT'd FFI, so there is no need to eg compile extensions for 4 different CPython ABIs (Julia also has syntactic macros which can be helpful here).
To provide a better experience for both package authors and users, as well as reducing the maintenance burden, the community has developed and migrated to a unified system called BinaryBuilder (https://binarybuilder.org) over the past 2-3 years. BinaryBuilder allows targeting all supported platforms with a single build script and also "audits" build products for common compatibility and linkage snafus (similar to some of the conda-build tooling and auditwheel). There was a nice talk at AlpineConf recently (https://alpinelinux.org/conf/) covering some of this history and detailing BinaryBuilder, although I'm not sure how to link into the video.
All that to say: it can work to an extent, but it has been tried various times before. The fact that conda and manylinux don't use system packages was not borne out of inexperience, either. The idea of "make binaries a distro packager's problem" sounds like a simplifying step, but that doesn't necessarily work out.
What are some alternatives?
jax - Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more
Yggdrasil - Collection of builder repositories for BinaryBuilder.jl
nn - 🧑🏫 60 Implementations/tutorials of deep learning papers with side-by-side notes 📝; including transformers (original, xl, switch, feedback, vit, ...), optimizers (adam, adabelief, sophia, ...), gans(cyclegan, stylegan2, ...), 🎮 reinforcement learning (ppo, dqn), capsnet, distillation, ... 🧠
HTTP.jl - HTTP for Julia
onnx-simplifier - Simplify your onnx model
dh-virtualenv - Python virtualenvs in Debian packages
torch2trt - An easy to use PyTorch to TensorRT converter
RDKit - The official sources for the RDKit library
py2many - Transpiler of Python to many other languages
StarWarsArrays.jl - Arrays indexed as the order of Star Wars movies
vision - Datasets, Transforms and Models specific to Computer Vision
mxe - MXE (M cross environment)