Pushshift API VS Sketchpad

As an average user, you may not notice much difference in your browsing experience. For those using external moderation or accessibility tools, though Reddit has announced that tools/apps focusing on accessibility and moderation will not be negatively affected by the pricing changes, many of these tools are already provided by third-party apps that will be affected. Sites such as Unddit (for tracking post/comment edit history), which run on APIs like Pushshift, that are not quite covered under the vague criteria of being “legal, non-commercial, and helpful to mods,” are likely to no longer work as intended going forward.

Pushshift also supports other query parameters, such as specifying a time range using 'before' and 'after' or sorting the results by 'score', 'num_comments', etc. You can find the full list of query parameters in the Pushshift documentation: https://github.com/pushshift/api

The thing to look at is Pusshift: https://github.com/pushshift/api

Rule 3 says I can't link directly, but I can link to this. Pushshift is absolutely amazing for things like this. Just enter a url like http://api.pushshift.io/reddit/search/comment?q="\[START OF COMMENT TEXT]" and you should be able to find it quite easily.

In the meantime, the third party tool Pushshift can accomplish pretty much the same thing, just with some effort. It's a database of (almost) every comment and submission on Reddit, and you can filter the results based on things like the author and keywords in the body. You can write the queries yourself, but it's much easier to just use a webpage designed to make it easy. For example, if you wanted to find old KITTY pings, you could just enter "groupbot" in the author field, and "KITTY" in the "search term" of https://camas.unddit.com/. The result is this:

It has, https://github.com/pushshift/api code behind the api server here.

Also I'm gonna try and download all of the trans subreddits using this script https://github.com/Watchful1/Sketchpad/blob/master/postDownloader.py. hopefully I can get it working tomorrow.

Looks like there are also some unofficial, faster ways. But I don't know if they work: https://github.com/Watchful1/Sketchpad/blob/master/postDownloader.py

A while back I wrote a fairly popular script that used the pushshift api to find overlapping users between subreddits. This doesn't work anymore since the api is down, so I threw together an updated script that does the same thing using the subreddit dump files.

If you actually just want to level comments I have an example here https://github.com/Watchful1/Sketchpad/blob/master/load_top_level.py

I have a script here that lets you download a specific subreddit or users entire history using pushshift. It's a good example of how the url works and how to iterate through results based on timestamp. You can add a q=keyword parameter to filter to only submissions/comments matching a specific keyword. And you could remove the subreddit parameter if you want data from all of reddit.

I have an old script I wrote a long time ago to fetch only the top level comments in a thread here, which isn't quite what you're trying to do but should be a good example.

In addition to the dump files, pushshift offers an API with powerful filtering options. The main limitation is that it takes quite some time to download a substantial amount of data. If you have a use case that doesn't cleanly align to specific subreddits, take a look at my api download script here. Again I'm happy to work with you to build something for a specific use case.

There was a script posted at https://github.com/Watchful1/Sketchpad/blob/master/overlapCounter.py This does exactly what I need by using the pushift api, but seems too slow to work as a web app, and also I have no idea where to begin in converting the script to a web app.