chia-plotter VS cudf

The original Madmax plotter supports both K33 and K34, https://github.com/madMAx43v3r/chia-plotter

I didn’t use madmax for like 8 months so I don’t remember, but checkout the default number here https://github.com/madMAx43v3r/chia-plotter The thread number is depending on how many thread your cpu got. I think the more thread is better. Try to put thread = 36 and buckets = 256

I would suggest canceling and plotting with the Chia POS plotter. For testing, the Chia POS plotter should be fast enough. The madMAx plotter is generally faster but it has had issues plotting from the Chia Network client in the past. If you want to use the madMAx plotter, I would suggest using the standalone version directly from madMAx's github (which has a link to the github that does windows compiles in the install directions).

git clone https://github.com/madMAx43v3r/chia-plotter.git

No need to reverse engineer anything. It's explained in all the detail you need in the Chia Proof of Space Construction document. It's implemented in chiapos though I find madmax's implementation of the plotter to be easier to read: https://github.com/madMAx43v3r/chia-plotter/tree/master/include/chia

The article you mentioned said that all the questions about the madmax plotter has to updated on its official GitHub repo, https://github.com/madMAx43v3r/chia-plotter, so I made a copy of the same complained and updated it onto the official madmax plotter repo (and the link was given by the official chia repo).

Yes, see https://github.com/madMAx43v3r/chia-plotter for plotting with a ramdisk and https://github.com/ericaltendorf/plotman to automate everything.

The OP can try using the non-packaged version https://github.com/madMAx43v3r/chia-plotter

You can get around this by using the standalone version of madmax or setting your temp directory as your final directory and using a script to monitor and move .plot files to your final directory.

You can install madmax directly (or use the compiled windows binary). Disclaimer: neither are officially endorsed but I've used them with no issues.

cuDF Documentation

cudf – GPU DataFrame Library

The interesting thing about Polars is that it does not try to be a drop-in replacement to pandas, like Dask, cuDF, or Modin, and instead has its own expressive API. Despite being a young project, it quickly got popular thanks to its easy installation process and its “lightning fast” performance.

Perhaps the largest for package size is the NVIDIA developed rapids toolkit https://rapids.ai/ . Even still adding things like pandas and some geospatial tools, you rapidly end up with an image well over a gigabyte, despite following cutting edge best practice with docker and python.

Yes sure, that is how OpenMP does; but on the other side: you seem to already do some basic type inference, and building an AST, no? Then you know as well the size and type of your vectors, and can execute actions in parallel if there is enough data to be worth parallelizing. Is there anyone who don't want their code to execute faster if it is possible? Those that do work in big data domain do use threads and vectorized instructions without user having to type in any directive; just import different library. Example, numpy or numpy with cuda backend, or similar GPU accelerated libraries like cudf.

Not the answer about Ray: you could use rapids.ai. I'm using it for for dataframe manipulation on GPU

To put Data Analytics on GPU Steroids, Try RAPIDS cudf https://rapids.ai/

You can scope out https://rapids.ai/. Nvidia's AI toolkits. They have some handy notebooks to poke at to get you started.