dtale
viztracer
Our great sponsors
dtale | viztracer | |
---|---|---|
46 | 5 | |
4,550 | 4,363 | |
2.2% | - | |
8.1 | 7.7 | |
1 day ago | 2 days ago | |
TypeScript | Python | |
GNU Lesser General Public License v3.0 only | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
dtale
-
The free pandas visualizer, D-Tale, has now been integrated with ArcticDB which will allow users to load huge datasets and easily navigate their databases
[D-Tale](https://github.com/man-group/dtale) has recently released version 3.2.0 on pypi & conda-forge: ``` pip install -U dtale conda install dtale -c conda-forge ``` But if you want to take it one step further you can now integrate it with [ArcticDB](https://github.com/man-group/ArcticDB): ``` pip install -U dtale[arcticdb] ``` This allows you the ability to navigate your libraries of datasets saved to your ArcticDB database! But the best part is that all the reads are occuring directly against ArcticDB so some of the memory constraints you may have been hit with before are now a thing of the past. Here's a full write up how to use this functionality along with a quick demo: https://github.com/man-group/dtale/blob/master/docs/arcticdb/ARCTICDB\_INTEGRATION.md Hope this helps & please support open-source by throwing your star on the [repo](https://github.com/man-group/dtale). Thanks! 🙏
-
Data Scientists using neovim: how do you explore dataframes?
I've looked into external tooling, libs such as dtale, which feel overly complicated for my use case (but I'm open to alternatives). What I would like to have instead is something akin to Spyder's variable viewer, which allows sorting by column. VSCode goes a step further and also provides the ability to filter the dataframe.
-
I need help lol
D-Tale: A Python library that provides an interactive web-based interface for data exploration and analysis.
-
Something better than pandas? with interactive graphical UI?
Try this: https://github.com/man-group/dtale
-
Mito – Excel-like interface for Pandas dataframes in Jupyter notebook
https://github.com/man-group/dtale
I find that I'm actually a lot faster using basic Pandas methods to get the data I want in exactly the form I want it.
If I really want to show everything, I just use:
'''
- Memray is a memory profiler for Python by Bloomberg
- Show HN: D-Tale, easy to use pandas GUI
-
Added visualizations of statsmodels time series analysis functions to the free pandas visualizer, D-Tale
Just added "Time Series Analysis" in v1.60.1 of D-Tale on pypi & conda-forge: pip install -U dtale conda install dtale -c conda-forge This feature provides a quick and easy way to visualize the usage of the following time series analysis function in statsmodels:
- Show HN: Open-source pandas dataframe visualizer
-
For all the python/pandas users out there I just released a bunch of UI updates to the free visualizer, D-Tale
Your data is stored in memory so the size of your dataframe is limited to the memory of your machine. That being said we’ve allowed users to swap out the machanism which stores the data so you can use something like Redis or Shelve to allieviate memory. Here’s some documentation: https://github.com/man-group/dtale/blob/master/docs/GLOBAL_STATE.md
viztracer
-
Ask HN: C/C++ developer wanting to learn efficient Python
* https://github.com/gaogaotiantian/viztracer get a timeline of execution vs call-stack (great to discover what's happening deep inside pandas)
-
GCC Profiler Internals
Do not use bad instrumenting profilers. A good modern tracing-based instrumenting profiler provides so much more actionable information and insights into where problems are than a sampling profiler it is ridiculous.
As a example consider viztracer [1] for Python. By using a aggregate visualizer such as a flame graph you can figure out what is taking the most time then you can use a tracing visualizer to figure out the exact call stacks and system execution and state that caused it. Not only that, a tracing visualizer lets you diagnose whole system performance and makes it trivial to identify 1 in 1000 anomalous execution patterns (with a 4k screen a anomalous execution pattern stands out like a 4 pixel dead spot). In addition you also get vastly less biased information for parallel execution and get easy insights into parallel execution slowdowns, interference, contention, and blocking behaviors.
The only advantages highlighted in your video that still apply to a good instrumenting profiler are:
1. Multi-language support.
2. Performance counters (though that is solved by doing manual tracking after you know the hotspots and causes).
3. Overhead (if you are using low sampling frequency). Even then a good tracing instrumentation implementation should only incur low double-digit percent overhead and maybe 100% overhead in truly pathological cases involving only small functions where the majority of the execution time is literally spent in function call overhead.
4. No need for recompilation, but you are already looking to make performance changes and test so you already intend to rebuild frequently to test those experiments. In addition, the relative difference in information is so humongous that this is not even worth contemplating unless it is a hard requirement like evaluating something in the field.
[1] https://github.com/gaogaotiantian/viztracer
-
Memray is a memory profiler for Python by Bloomberg
Actually it has explicit support for async task based reporting:
https://github.com/gaogaotiantian/viztracer#async-support
-
Tracing and visualizing the Python GIL with perf and VizTracer
Let us run perf on this, similarly to what we did to example0.py. However, we add the argument -k CLOCK_MONOTONIC so that we use the same clock as VizTracer and ask VizTracer to generate a JSON, instead of an HTML file:
What are some alternatives?
PandasGUI - A GUI for Pandas DataFrames
pytest-austin - Python Performance Testing with Austin
ydata-profiling - 1 Line of code data quality profiling & exploratory data analysis for Pandas and Spark DataFrames.
magic-trace - magic-trace collects and displays high-resolution traces of what a process is doing
jupyterlab-autoplot - Magical Plotting in JupyterLab
scalene - Scalene: a high-performance, high-precision CPU, GPU, and memory profiler for Python with AI-powered optimization proposals
pandastable - Table analysis in Tkinter using pandas DataFrames.
gil_load - Utility for measuring the fraction of time the CPython GIL is held
sqliteviz - Instant offline SQL-powered data visualisation in your browser
memray - Memray is a memory profiler for Python
best-of-ml-python - 🏆 A ranked list of awesome machine learning Python libraries. Updated weekly.
Apache Arrow - Apache Arrow is a multi-language toolbox for accelerated data interchange and in-memory processing