Coconut VS functorch

> to be part of the Python ecosystem

I'd rather use Python if I'm in the Python ecosystem. So many attempts were made in the past to make a new language compatible with the Python ecosystem (look up hylang and coconu -- https://github.com/evhub/coconut). But at the end of the day, I'd come back to Python because if there's one thing I've learnt in recent years it's this:

    minimize dependencies at all costs.

Have you had a look at Coconut? I don't know if it'll push all your buttons but whenever I hear someone who's reasonably content with Python but wants more FP goodies I always think of it. https://github.com/evhub/coconut . It's basically a superset of Python3 that transpiles into Python3 and is compatible with MyPy. I don't think I'd code Python w/o it ever again assuming I had the choice. The biggest negative for me is that there's no IDE support for the language last I looked, though of course you can work with the transpiler output (plain Python) in your favorite Python IDE. It might be fun to play around with, I know that I really enjoyed it but then I got spoiled by the language+tooling of Scala3, but if you don't have that option ...

If you're looking for suggestions I want to put forward zig lang if you like C/C++ and Coconut Lang if you like Python!

F# and also the fun, compile-to-Python, functional language called Coconut.

I would suggest checking https://pytorch.org/functorch/ and https://github.com/metaopt/torchopt for efficient inference and training with ensembles (e.g., t be independent actors in a multi-agent setting or multiple critics).

There were some bugs still with advanced indexing in an older release of functorch, I believe they should be fixed now though: https://github.com/pytorch/functorch/pull/862

Tangentially related but there is an effort to get some of the features of JAX into PyTorch: https://pytorch.org/functorch/

Fwiw, composable vmap and stuff like that have also been implemented in PyTorch now - see functorch :) https://github.com/pytorch/functorch

Fwiw, it’s not like Pytorch’s design prevents function transformations from being implemented. See functorch for an example of grad/vmap function transforms: https://github.com/pytorch/functorch

You may find vmap from functorch to be quite useful: https://github.com/pytorch/functorch

If you're interested in function transformations in PyTorch, try out functorch :) https://github.com/pytorch/functorch

The pytorch programming model is just really hard to adapt to an XLA-like compiler. Imperative python code doesn't translate to an ML graph compiler particularly well; Jax's API is functional, so it's easier to translate to the XLA API. By contrast, torch/xla uses "lazy tensors" that record the computation graph and compile when needed. The trouble is, if the compute graph changes from run to run, you end up recompiling a lot.

I guess in Jax you'd just only apply `jax.jit` to the parts where the compute graph is static? I'd be curious to see examples of how this works in practice. Fwiw, there's an offshoot of pytorch that is aiming to provide this sort of API (see https://github.com/pytorch/functorch and look at eager_compilation.py).

(Disclaimer: I worked on this until quite recently.)