elegy VS long-range-arena

I wonder if https://github.com/poets-ai/elegy is still an active project or dead because it hasn't had a commit in almost a year. Would be too bad if abandoned because I like it.

Elegy might be worth a look.

JAX is really cool, but still somewhat immature. I would love to see it taking more ground and improving wrt e.g. integration with tensorboard and getting all the goodies we have in tensorflow. If you are looking for a higher level framework, I would recommend elegy [0] which is very close to the keras API.

[0] https://github.com/poets-ai/elegy

What's your favorite Deep Learning API for JAX - Flax, Haiku, Elegy, something else?

For a Module library checkout Flax or Haiku, they are well maintained. For a Trainer interface like Keras / Pytorch Lightning checkout Elegy: https://github.com/poets-ai/elegy

https://github.com/google-research/long-range-arena

The Structured State Space for Sequence Modeling (S4) architecture is a new approach to very long-range sequence modeling tasks for vision, language, and audio, showing a capacity to capture dependencies over tens of thousands of steps. Especially impressive are the model’s results on the challenging Long Range Arena benchmark, showing an ability to reason over sequences of up to 16,000+ elements with high accuracy.

1.1 Long Range Arena: A Benchmark for Efficient Transformers. From authors of above, they proposed a benchmark for modeling long range interactions. It also inlcudes a repository

I think the paper is written in a clear style and I like that the authors included many experiments, including hyperparameter effects, ablations and extensive baseline comparisons. One thing I would have liked is them comparing their Informer to more efficient transformers (they compared only against logtrans and reformer) using the LRA (https://github.com/google-research/long-range-arena) benchmark.