StarWarsArrays.jl VS functorch

I raise you https://github.com/giordano/StarWarsArrays.jl

Why not Star Wars Indices (4,5,6,1,2,3,7,8,9...)? https://github.com/giordano/StarWarsArrays.jl

> The documentation disagrees about string indices not starting with 1 As priorly said, I'm speaking about strings, not `String` in particular. So, to write code which work for all AbstractString (which have basic string functions), you must not assume that the first indexing is 1, you can have degenerate cases such as : https://github.com/giordano/StarWarsArrays.jl (this is for vectors, but creating a similar type, for AbstractString isn't impossible) or just strings with an offset indexing.

There are other choices like https://github.com/simonster/TwoBasedIndexing.jl and https://github.com/giordano/StarWarsArrays.jl if you do not like 1-based indexing.

This is a total non issue as indexing is an operation that is subject to multiple dispatch. For a humorous example see https://github.com/giordano/StarWarsArrays.jl

The cool thing with Julia is that array indices aren't inherent properties, and may be changed locally by using appropriate wrappers. This means that the same underlying array may start at 0 in one part of the code, at 1 in another, and perhaps use the star-wars indexing in yet another section if that's necessary.

Adding https://github.com/giordano/StarWarsArrays.jl to the list for some extra spice

You should also check out https://github.com/giordano/StarWarsArrays.jl and https://github.com/giordano/RandomBasedArrays.jl

I would suggest checking https://pytorch.org/functorch/ and https://github.com/metaopt/torchopt for efficient inference and training with ensembles (e.g., t be independent actors in a multi-agent setting or multiple critics).

There were some bugs still with advanced indexing in an older release of functorch, I believe they should be fixed now though: https://github.com/pytorch/functorch/pull/862

Tangentially related but there is an effort to get some of the features of JAX into PyTorch: https://pytorch.org/functorch/

Fwiw, composable vmap and stuff like that have also been implemented in PyTorch now - see functorch :) https://github.com/pytorch/functorch

Fwiw, it’s not like Pytorch’s design prevents function transformations from being implemented. See functorch for an example of grad/vmap function transforms: https://github.com/pytorch/functorch

You may find vmap from functorch to be quite useful: https://github.com/pytorch/functorch

If you're interested in function transformations in PyTorch, try out functorch :) https://github.com/pytorch/functorch

The pytorch programming model is just really hard to adapt to an XLA-like compiler. Imperative python code doesn't translate to an ML graph compiler particularly well; Jax's API is functional, so it's easier to translate to the XLA API. By contrast, torch/xla uses "lazy tensors" that record the computation graph and compile when needed. The trouble is, if the compute graph changes from run to run, you end up recompiling a lot.

I guess in Jax you'd just only apply `jax.jit` to the parts where the compute graph is static? I'd be curious to see examples of how this works in practice. Fwiw, there's an offshoot of pytorch that is aiming to provide this sort of API (see https://github.com/pytorch/functorch and look at eager_compilation.py).

(Disclaimer: I worked on this until quite recently.)