papermill VS jupyter-vim

Pandas docs > Comparison with spreadsheets: https://pandas.pydata.org/docs/getting_started/comparison/co...

Pandas docs > I/O > Excel files: https://pandas.pydata.org/docs/user_guide/io.html#excel-file...

nteract/papermill: https://github.com/nteract/papermill :

> papermill is a tool for parameterizing, executing, and analyzing Jupyter Notebooks. [...]

> This opens up new opportunities for how notebooks can be used. For example:

> - Perhaps you have a financial report that you wish to run with different values on the first or last day of a month or at the beginning or end of the year, using parameters makes this task easier.

"The World Excel Championship is being broadcast on ESPN" (2022) https://news.ycombinator.com/item?id=32420925 :

> Computational notebook speedrun ideas:

There is Papermill ... https://github.com/nteract/papermill

https://github.com/jupyter/enhancement-proposals/pull/103#is...

Papermill is one tool for running Jupyter notebooks as reports; with the date in the filename. https://papermill.readthedocs.io/en/latest/

You may be interested in papermill to address the parametrized analysis problem [1]. I think (but I'm not positive) this is what the data team at a previous job used to automate running notebooks for all sorts nightly reports.

[1] https://papermill.readthedocs.io/en/latest/#

I'm using Papermill to operationalize Notebooks (https://github.com/nteract/papermill), it e.g. also has airflow support. I'm really happy with papermill for automatic notebook execution, in my field it's nice that we can go very quickly from analysis to operations -- while having super transparent "logging" in the executed notebooks.

papermill bcpandas fastapi

But you can execute your notebook using Jupyter-run or papermill.

As a first option, we will use Papermill, which has a Python API that allows us to run different notebooks using some functions:

Papermill can also target cloud storage outputs for hosting rendered notebooks, execute notebooks from custom Python code, and even be used within distributed data pipelines like Dagster (see Dagstermill). For more information, see the papermill documentation.

Papermill Source Code

In this config, under the Plugins folder there are various plugins that aren't found in the Astrocommunity repo (aka "Custom" plugins). A great example that I found interesting and included into my own config was mehalters repl.lua file that installs the jupyter-vim plugin. From my limited understanding to install packages into astrovim, we need to make a file in the Plugins folder, make the necessary configurations, and then it should load.

Is there any way to communicate with jupyter like jupyter-vim? I am aware of python-repl in emacs, but it does not support inline plots/figures. Moreover, it is nice to nice to show outputs in different windows (especially using multiple monitors).

Now here comes the endboss: Jupyter. For the first time, I feel like I'm missing out on stuff when using vim. I've started a job in datascience, which is actually awesome. However, I work a lot with image data. I also do a lot of analysis on results, meaning I do a lot of fancy plots that hopefully show the weaknesses of our prediction models. I recently wrote an augmentation algorithm where I had to see the output in form of an image after every step to make sure it's correct. This is not a possible workflow in vim right now. I know of many solutions that I already tried, like for example jupyter-vim or the jupyter vim mode. I'd like to work inside my terminal though. I'm not this kind of purist who needs to have a terminal that is compatible with VT100 or whatever people came up with in the 80ies. I also don't care if my terminal in based on an ascii like grid or actually rendered in HTML. I just want (Neo)vim, with the functionality of jupyter (inline plotting) even if this means vim has to be rendered inside an electron app or whatever people use these days for fancy GUIs. Imagine an electron based editor like Oni which not only runs the "real" neovim in the background, but is also able to do inline figures, images, plots and even interactive stuff. It seems to me like I can't be the only one who wants this. So after all this, here's the question: Is there anything you now of that allows for this kind of stuff? Is there any other workflow that I'm not aware of? Or do people just not use those features when working with vim? Pls help a vimmer stay at vim.

For Matlab-like "cell-mode", have a look at https://github.com/jupyter-vim/jupyter-vim/.

To work in a py file (or py codelines selection from whatever format): https://github.com/jupyter-vim/jupyter-vim