mercury VS papermill

Web App is using two open-source packages that I've created:

- MLJAR AutoML - Python package for AutoML on tabular data https://github.com/mljar/mljar-supervised

- Mercury - framework for converting Jupyter Notebooks into Web App https://github.com/mljar/mercury

You can run Web App locally. What is more, you can adjust notebook's code for your needs. For example, you can set different validation strategies or evalutaion metrics or longer training times. The notebooks in the repo are good starting point for you to develop more advanced apps.

The GitHub repository https://github.com/mljar/mercury

We are working on open-source web framework Mercury that converts Python notebooks to Web Apps.

It is very opinionated:

- it has no callbacks - we automatically re-execute cells below updated widget

- it has no layout widgets, all input widgets are always in the left sidebar

Thanks to above decisions you don't need to change notebook's code to have web app and fit to the framework.

The simplicity of the framework is very important to us. We also care about deployment simplicity. That's why we created a shared hosting service called Mercury Cloud. You can deploy notebook by uploading a file.

The GitHub repository https://github.com/mljar/mercury

Documentation https://RunMercury.com/docs/

Mercury Cloud https://cloud.runmercury.com

Analysis was done in Jupyter Notebook with Python 3.10, Pandas, Matplotlib, wordcloud and Mercury framework.

Analysis was done in Jupyter Notebook with Python 3.10 kernel, Pandas, Matplotlib, wordcloud and Mercury framework to share notebook as a web application with widgets and code hidden. Gif created in Canva.

Mercury is a framework for converting Jupyter Notebooks to interactive apps (Voila killer),

Pandas docs > Comparison with spreadsheets: https://pandas.pydata.org/docs/getting_started/comparison/co...

Pandas docs > I/O > Excel files: https://pandas.pydata.org/docs/user_guide/io.html#excel-file...

nteract/papermill: https://github.com/nteract/papermill :

> papermill is a tool for parameterizing, executing, and analyzing Jupyter Notebooks. [...]

> This opens up new opportunities for how notebooks can be used. For example:

> - Perhaps you have a financial report that you wish to run with different values on the first or last day of a month or at the beginning or end of the year, using parameters makes this task easier.

"The World Excel Championship is being broadcast on ESPN" (2022) https://news.ycombinator.com/item?id=32420925 :

> Computational notebook speedrun ideas:

There is Papermill ... https://github.com/nteract/papermill

https://github.com/jupyter/enhancement-proposals/pull/103#is...

Papermill is one tool for running Jupyter notebooks as reports; with the date in the filename. https://papermill.readthedocs.io/en/latest/

You may be interested in papermill to address the parametrized analysis problem [1]. I think (but I'm not positive) this is what the data team at a previous job used to automate running notebooks for all sorts nightly reports.

[1] https://papermill.readthedocs.io/en/latest/#

I'm using Papermill to operationalize Notebooks (https://github.com/nteract/papermill), it e.g. also has airflow support. I'm really happy with papermill for automatic notebook execution, in my field it's nice that we can go very quickly from analysis to operations -- while having super transparent "logging" in the executed notebooks.

papermill bcpandas fastapi

But you can execute your notebook using Jupyter-run or papermill.

As a first option, we will use Papermill, which has a Python API that allows us to run different notebooks using some functions:

Papermill can also target cloud storage outputs for hosting rendered notebooks, execute notebooks from custom Python code, and even be used within distributed data pipelines like Dagster (see Dagstermill). For more information, see the papermill documentation.

Papermill Source Code