teddit VS Pushshift API

Someone should just convert the Teddit UI into a Lemmy theme.

https://github.com/teddit-net/teddit

Why scrapy if you have teddit? These kinds of projects are the best tbh

Not really everyone's cup of tea but as an interesting aside... I've recently become quite enamoured with the various "alternative frontends" that are floating around, like invidious for youtube or teddit for reddit. These are not just about blocking ads, (although they do that), they also make these platforms less hostile to your mental health.

Teddit is the equivalent of Invidious but just like Invidious you can't log in, it would beat the purpose if you could. Most of that TOS would still apply if you are logged trough an app or front end. tracking you can minimize by opting out and using with uBlock Origin.

The archiving began in earnest when I started limiting my time exlusively to the FTF. Before that I focused more on archiving fanart, fanfiction and mods for a later use I still want to get to at some point. My archives of FTF threads are HTML files downloaded though the teddit frontend(https://github.com/teddit-net/teddit). Though my archive is much bigger than that. To fill in the backlog of posts from before I started archiving manually I used a Python script to scrape the reddit pushshift archive https://github.com/pushshift/api in order to make sure that I grabbed every comment said in any thread created by JustMonika as well her post text though this backlog data is not easily browsable in its current form, its a whole lot of JSON.

Before writing something from scratch, look at a load of open source projects that do parts of what you need, like: https://github.com/teddit-net/teddit. But there are loads more to research.

Teddit.net[0] solves a lot of these problems and paired with Redirector[1], it's a transparent switch out.

https://github.com/teddit-net/teddit

[2]: https://github.com/teddit-net/teddit

As an average user, you may not notice much difference in your browsing experience. For those using external moderation or accessibility tools, though Reddit has announced that tools/apps focusing on accessibility and moderation will not be negatively affected by the pricing changes, many of these tools are already provided by third-party apps that will be affected. Sites such as Unddit (for tracking post/comment edit history), which run on APIs like Pushshift, that are not quite covered under the vague criteria of being “legal, non-commercial, and helpful to mods,” are likely to no longer work as intended going forward.

Pushshift also supports other query parameters, such as specifying a time range using 'before' and 'after' or sorting the results by 'score', 'num_comments', etc. You can find the full list of query parameters in the Pushshift documentation: https://github.com/pushshift/api

The thing to look at is Pusshift: https://github.com/pushshift/api

Rule 3 says I can't link directly, but I can link to this. Pushshift is absolutely amazing for things like this. Just enter a url like http://api.pushshift.io/reddit/search/comment?q="\[START OF COMMENT TEXT]" and you should be able to find it quite easily.

In the meantime, the third party tool Pushshift can accomplish pretty much the same thing, just with some effort. It's a database of (almost) every comment and submission on Reddit, and you can filter the results based on things like the author and keywords in the body. You can write the queries yourself, but it's much easier to just use a webpage designed to make it easy. For example, if you wanted to find old KITTY pings, you could just enter "groupbot" in the author field, and "KITTY" in the "search term" of https://camas.unddit.com/. The result is this:

It has, https://github.com/pushshift/api code behind the api server here.