cloud-to-butt VS Pushshift-Importer

We need an AI version of the cloud-to-butt browser extension.

Additionally, there's an open-source chrome plugin that does this exact same thing, except it replaces the word "cloud" with the word "butt": https://github.com/panicsteve/cloud-to-butt. If my object-regex idea is dumb, you can look at this extension to see how it should really be done.

TFA:

> Inspired by and adapted from the infamous [cloud-to-butt](https://github.com/panicsteve/cloud-to-butt) extension, as originally released under the WTFPL (here BSD0)

Not a bot, but you can modify this extension to do it on your computer. https://github.com/panicsteve/cloud-to-butt

In fact it's on GitHub apparently so it would have absolutely been outed. https://github.com/panicsteve/cloud-to-butt

Plus implies it's not the original.

The original started in 2013: https://github.com/panicsteve/cloud-to-butt/commit/c518a1317...

Rip off of Cloud to Butt.

https://github.com/panicsteve/cloud-to-butt

Download source for Cloud to Butt

Cloud to butt, appears to just edit the html before displaying it in your browser. I don't think that will work for CSV files (now that I think of it I don't think you can open csv in a browser to begin with ?). If you just want to duplicate the way cloud-to-butt works, it appears that you can just edit the regex-replace at the bottom of the code.

I'm thinking i will have to get a project like redarc or BDFR-to-HTML or much more likely Pushshift-Importer which allows you to import pushshift downloads into a SQLite database. From there i would have to hook up the database to a reddit-like frontend.

Just download the dumps from pushshift and then use Pushshift-Importer.

I wrote my own rust based importer. Feel free to use types and such from that as well.

I built a command line tool to import the dumps into sqlite if you want to give it a go. https://github.com/Paul-E/Pushshift-Importer

I wrote some code to do just this. Input the locations of the comments and submissions and it will produce an output sqlite file.

I created a pushshift importer for comments. You can find it here. It will import the comments into a sqlite database. It is written in rust and is very fast compared to python. It can import everything overnight if you have an SSD.

If you stick with SQLite, you could try creating your own sequencer. Funnel all your writes into one thread on one process, and have that thread do the writing. That way there is only ever one possible writer on the DB at a time. Here is an example what I did when I built a tool to import comments from pushshift into SQLite. When I do this on an NVME drive and I am CPU bound on decompression and JSON parsing, so the DB isn't even a bottleneck.