livebook VS explorer

To get started you need a running instance of Livebook

LiveBook site

I prefer functional programming with Livebook[1] for this type of thing. Once you run a cell, it can be published right into a web component as well.

[1] - https://livebook.dev

If you're a coder, Livebook might be worth a look too. I certainly have my eyes on it.

Would highly recommend looking at Jose's use of livebook to answer these. It makes testing easier. It's old but still relevant. Video link inside

That's hard to say. Racket is a rather complete language, as is F# and Elixir. And F# and Racket are extremely capable multi-paradigm languages, supporting basically any paradigm. Elixir is a bit more restricted in terms of its paradigms, but that's a feature oftentimes, and it also makes up for it with its process framework and deep VM support from the BEAM.

I would say that the key difference is that F# and Elixir are backed by industry whereas Racket is primarily backed via academia. Thus, the incentives and goals are more aligned for F# and Elixir to be used in industrial settings.

Also, both F# and Elixir gain a lot from their host VMs in the CLR and BEAM. Overall, F# is the cleanest language of the three, as it is easy to write concise imperative, functional, or OOP code and has easy asynchronous facilities. Elixir supports macros, and although Racket's macro system is far more advanced, I don't think it really provides any measurable utility over Elixir's. I would also say that F# and Elixir's documentation is better than Racket's. Racket has a lot of documentation, but it can be a little terse at times. And Elixir definitely has the most active, vibrant, and complete ecosystem of all three languages, as well as job market.

The last thing is that F# and Elixir have extremely good notebook implementations in Polyglot Notebooks (https://marketplace.visualstudio.com/items?itemName=ms-dotne...) and Livebook (https://livebook.dev/), respectively. I would say both of these exceed the standard Python Jupyter notebook, and Racket doesn't have anything like Polyglot Notebooks or Livebook. (As an aside, it's possible for someone to implement a Racket kernel for Polyglot Notebooks, so maybe that's a good side project for me.)

So for me, over time, it has slowly whittled down to F# and Elixir being my two languages that I reach for to handle effectively any project. Racket just doesn't pull me in that direction, and I would say that Racket is a bit too locked to DrRacket. I tried doing some GUI stuff in Racket, and despite it having an already built framework, I have actually found it easier to write my own due to bugs found and the poor performance of Racket Draw.

This looks very similar to LiveBook¹. It is purely Elixir/BEAM based, but is quite polished and seems like a perfect workflow tool that is also able to expose these workflows (simply called livebooks) as web apps that some functional, non-technical person can execute on his/her own.

1: https://livebook.dev/

The Explorer library [0] in Elixir uses Polars underneath it.

[0] https://github.com/elixir-explorer/explorer

To ensure you do not miss this: LiveBook comes with a Vega Lite integration (https://livebook.dev/integrations -> https://livebook.dev/integrations/vega-lite/), which means you get access to a lot of visualisations out of the box, should you need that (https://vega.github.io/vega-lite/).

In the same "standing on giant's shoulders" stance, you can use Explorer (see example LiveBook at https://github.com/elixir-explorer/explorer/blob/main/notebo...), which leverages Polars (https://www.pola.rs), a very fast DataFrame library and now a company (https://www.pola.rs/posts/company-announcement/) with 4M$ seed.

Already exists. Check out https://github.com/elixir-nx/explorer which provides a tidyverse-like API in Elixir using polars as the back end.

José from the Livebook team. I don't think I can make a pitch because I have limited Python/R experience to use as reference.

My suggestion is for you to give it a try for a day or two and see what you think. I am pretty sure you will find weak spots and I would be very happy to hear any feedback you may have. You can find my email on my GitHub profile (same username).

In general we have grown a lot since the Numerical Elixir effort started two years ago. Here are the main building blocks:

* Nx (https://github.com/elixir-nx/nx/tree/main/nx#readme): equivalent to Numpy, deeply inspired by JAX. Runs on both CPU and GPU via Google XLA (also used by JAX/Tensorflow) and supports tensor serving out of the box

* Axon (https://github.com/elixir-nx/axon): Nx-powered neural networks

* Bumblebee (https://github.com/elixir-nx/bumblebee): Equivalent to HuggingFace Transformers. We have implemented several models and that's what powers the Machine Learning integration in Livebook (see the announcement for more info: https://news.livebook.dev/announcing-bumblebee-gpt2-stable-d...)

* Explorer (https://github.com/elixir-nx/explorer): Series and DataFrames, as per this thread.

* Scholar (https://github.com/elixir-nx/scholar): Nx-based traditional Machine Learning. This one is the most recent effort of them all. We are treading the same path as scikit-learn but quite early on. However, because we are built on Nx, everything is derivable, GPU-ready, distributable, etc.

Regarding visualization, we have "smart cells" for VegaLite and MapLibre, similar to how we did "Data Transformations" in the video above. They help you get started with your visualizations and you can jump deep into the code if necessary.

I hope this helps!

There's a package in the Nx ecosystem called Explorer (https://github.com/elixir-nx/explorer). It uses bindings for the rust library, polars, which is much more betterer than Pandas.

FWIW around April this year I started using erlport with python polars in a production ETL app because explorer didn't have the features I needed at the time.

Machine learning is rapidly expanding within the Elixir ecosystem, with tools such as Nx, Axon, and Explorer being used both by individuals and companies such as Amplified, as mentioned above.