papermill
ipython
papermill | ipython | |
---|---|---|
26 | 34 | |
5,630 | 16,135 | |
0.6% | 0.1% | |
8.0 | 9.6 | |
6 days ago | 5 days ago | |
Python | Python | |
BSD 3-clause "New" or "Revised" License | BSD 3-clause "New" or "Revised" License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
papermill
-
Spreadsheet errors can have disastrous consequences â yet we keep making them
Pandas docs > Comparison with spreadsheets: https://pandas.pydata.org/docs/getting_started/comparison/co...
Pandas docs > I/O > Excel files: https://pandas.pydata.org/docs/user_guide/io.html#excel-file...
nteract/papermill: https://github.com/nteract/papermill :
> papermill is a tool for parameterizing, executing, and analyzing Jupyter Notebooks. [...]
> This opens up new opportunities for how notebooks can be used. For example:
> - Perhaps you have a financial report that you wish to run with different values on the first or last day of a month or at the beginning or end of the year, using parameters makes this task easier.
"The World Excel Championship is being broadcast on ESPN" (2022) https://news.ycombinator.com/item?id=32420925 :
> Computational notebook speedrun ideas:
-
Jupyter Kernel Architecture
There is Papermill ... https://github.com/nteract/papermill
-
Git and Jupyter Notebooks Guide
https://github.com/jupyter/enhancement-proposals/pull/103#is...
Papermill is one tool for running Jupyter notebooks as reports; with the date in the filename. https://papermill.readthedocs.io/en/latest/
-
JupyterLab 4.0
You may be interested in papermill to address the parametrized analysis problem [1]. I think (but I'm not positive) this is what the data team at a previous job used to automate running notebooks for all sorts nightly reports.
[1] https://papermill.readthedocs.io/en/latest/#
-
Show HN: Mercury â convert Jupyter Notebooks to Web Apps without code rewriting
I'm using Papermill to operationalize Notebooks (https://github.com/nteract/papermill), it e.g. also has airflow support. I'm really happy with papermill for automatic notebook execution, in my field it's nice that we can go very quickly from analysis to operations -- while having super transparent "logging" in the executed notebooks.
-
What's the best thing/library you learned this year ?
papermill bcpandas fastapi
-
Does the Jupyter API allow using Jupyter from the CL?
But you can execute your notebook using Jupyter-run or papermill.
-
Running Jupyter notebooks in parallel
As a first option, we will use Papermill, which has a Python API that allows us to run different notebooks using some functions:
-
Tips for using Jupyter Notebooks with GitHub
Papermill can also target cloud storage outputs for hosting rendered notebooks, execute notebooks from custom Python code, and even be used within distributed data pipelines like Dagster (see Dagstermill). For more information, see the papermill documentation.
-
Three Tools for Executing Jupyter Notebooks
Papermill Source Code
ipython
-
The new pdbp (Pdb+) Python debugger!
If youâre already using ipython, this isnât a problem because youâll already need to download most of these dependencies anyway. But if youâre not using ipython⌠youâll still need to download those dependencies.
-
Pandas 1.5 released
!pip install is error-prone, it is better to use %pip install, ipython even warns about this, https://github.com/ipython/ipython/pull/12954/
-
Why deprecate loading unpackaged extensions?
The git history (here is the git blame) shows it has not been updated in 9 year. Looks like a documentation issue that you should open an issue against.
-
Debugging Python programs without an IDE
Do you know IPython? It is a modern Python console that extends the capabilities of the classic builtin Python shell by offering introspection, tab completion, syntaxing coloring, as well as history. If you don't know it, I can't recommend it enough. More information can be found in its GitHub page.
-
External variables in lambda functions in Python
There is an IPython ticket on GitHub on the topic, but it's unclear if the problem has been solved.
-
Pipx: A python package consumption tool for CLI packages
For further documentation on ipython using the CLI, you can refer to the GitHub link or the documentation page.
-
Workflow-killing crash from strange added characters.
> ??????_ Traceback (most recent call last): File "/home/nvaughn4/bin/miniconda3/envs/newprime/bin/ipython", line 11, in sys.exit(start_ipython()) File "/home/nvaughn4/bin/miniconda3/envs/newprime/lib/python3.6/site-packages/IPython/__init__.py", line 126, in start_ipython return launch_new_instance(argv=argv, **kwargs) File "/home/nvaughn4/bin/miniconda3/envs/newprime/lib/python3.6/site-packages/traitlets/config/application.py", line 658, in launch_instance app.start() File "/home/nvaughn4/bin/miniconda3/envs/newprime/lib/python3.6/site-packages/IPython/terminal/ipapp.py", line 356, in start self.shell.mainloop() File "/home/nvaughn4/bin/miniconda3/envs/newprime/lib/python3.6/site-packages/IPython/terminal/interactiveshell.py", line 563, in mainloop self.interact() File "/home/nvaughn4/bin/miniconda3/envs/newprime/lib/python3.6/site-packages/IPython/terminal/interactiveshell.py", line 554, in interact self.run_cell(code, store_history=True) File "/home/nvaughn4/bin/miniconda3/envs/newprime/lib/python3.6/site-packages/IPython/core/interactiveshell.py", line 2858, in run_cell raw_cell, store_history, silent, shell_futures) File "/home/nvaughn4/bin/miniconda3/envs/newprime/lib/python3.6/site-packages/IPython/core/interactiveshell.py", line 2880, in _run_cell elif self.should_run_async(raw_cell): File "/home/nvaughn4/bin/miniconda3/envs/newprime/lib/python3.6/site-packages/IPython/core/interactiveshell.py", line 2918, in should_run_async return _should_be_async(cell) File "/home/nvaughn4/bin/miniconda3/envs/newprime/lib/python3.6/site-packages/IPython/core/async_helpers.py", line 161, in _should_be_async code = compile(cell, "<>", "exec") UnicodeEncodeError: 'utf-8' codec can't encode characters in position 537-542: surrogates not allowed If you suspect this is an IPython 7.15.0 bug, please report it at: https://github.com/ipython/ipython/issues or send an email to the mailing list at [email protected] You can print a more detailed traceback right now with "%tb", or use "%debug" to interactively debug it. Extra-detailed tracebacks for bug-reporting purposes can be enabled via: %config Application.verbose_crash=True sys:1: RuntimeWarning: coroutine 'InteractiveShell.run_cell_async' was never awaited
-
No_color
Thereâs one Iâve come across recently here where youâre fighting against syntax highlighting with extra error context. https://github.com/ipython/ipython/issues/13446#issuecomment...
-
Anybody else getting tired of parso and jedi?
I see. https://github.com/ipython/ipython/issues/13529
-
Django Codebase Reformatted with Black
You can automate setup for developers using this simple script:
https://github.com/ipython/ipython/pull/12091/files
And hereâs a GitLab issue requesting support for blame-ignore:
https://gitlab.com/gitlab-org/gitlab/-/issues/31423
I donât think thereâs a corresponding GitHub request, but maybe if GitLab adds this feature GitHub will have some incentive to follow suit.
What are some alternatives?
nbconvert - Jupyter Notebook Conversion
CPython - The Python programming language
ploomber - The fastest âĄď¸ way to build data pipelines. Develop iteratively, deploy anywhere. âď¸
bpython - bpython - A fancy curses interface to the Python interactive interpreter
airflow-notebook - This repository is no longer maintained.
xonsh - :shell: Python-powered, cross-platform, Unix-gazing shell.
nbdev - Create delightful software with Jupyter Notebooks
vim-slime - A vim plugin to give you some slime. (Emacs)
voila - VoilĂ turns Jupyter notebooks into standalone web applications
spacemacs - A community-driven Emacs distribution - The best editor is neither Emacs nor Vim, it's Emacs *and* Vim!
jupytext - Jupyter Notebooks as Markdown Documents, Julia, Python or R scripts
ptpython - A better Python REPL