DataProfiler VS XlsxWriter

It's been possible to skip tokenization for a long time, my team and I did it here - https://github.com/capitalone/DataProfiler

For what it's worth, we actually were working with LSTMs with nearly a billion params back in 2016-2017 area. Transformers made it far more effective to train and execute, but ultimately LSTMs are able to achieve similar results, though slow & require more training data.

Great call out -- would you be willing to write up an issue for that on the repo? Thank you! https://github.com/capitalone/DataProfiler/issues/new/choose

openpyxl

If you must, use xlsxwriter to write it directly into the Excel spreadsheet. This is kind of a pain. If you can get away with, say, writing the data to a CSV and then copy-pasting it into the Excel sheet, you should do that instead because it's so much more conceptually simple and works directly with the Python data structures that you parsed in Step 3. Automation doesn't have to be completely fire-and-forget to save you a ton of time!

https://xlsxwriter.readthedocs.io/ is what you are looking for most probably. Try using this library instead of pyplot for the above steps

xlsxwriter…

I've been in this situation a few times. I would just open the excel sheet in python using xlsxwriter: https://xlsxwriter.readthedocs.io/

Bam! https://xlsxwriter.readthedocs.io

For the opposite direction, I have had good luck with the XlsxWriter library

https://github.com/jmcnamara/XlsxWriter

I'm less familiar with the R ecosystem, but there are tools in the Python world to go that direction -- https://xlsxwriter.readthedocs.io/