spotifydb VS teddit

I wrote a npm package which can scrape the data some time ago, here it is.

Someone should just convert the Teddit UI into a Lemmy theme.

https://github.com/teddit-net/teddit

Why scrapy if you have teddit? These kinds of projects are the best tbh

Not really everyone's cup of tea but as an interesting aside... I've recently become quite enamoured with the various "alternative frontends" that are floating around, like invidious for youtube or teddit for reddit. These are not just about blocking ads, (although they do that), they also make these platforms less hostile to your mental health.

Teddit is the equivalent of Invidious but just like Invidious you can't log in, it would beat the purpose if you could. Most of that TOS would still apply if you are logged trough an app or front end. tracking you can minimize by opting out and using with uBlock Origin.

The archiving began in earnest when I started limiting my time exlusively to the FTF. Before that I focused more on archiving fanart, fanfiction and mods for a later use I still want to get to at some point. My archives of FTF threads are HTML files downloaded though the teddit frontend(https://github.com/teddit-net/teddit). Though my archive is much bigger than that. To fill in the backlog of posts from before I started archiving manually I used a Python script to scrape the reddit pushshift archive https://github.com/pushshift/api in order to make sure that I grabbed every comment said in any thread created by JustMonika as well her post text though this backlog data is not easily browsable in its current form, its a whole lot of JSON.

Before writing something from scratch, look at a load of open source projects that do parts of what you need, like: https://github.com/teddit-net/teddit. But there are loads more to research.

Teddit.net[0] solves a lot of these problems and paired with Redirector[1], it's a transparent switch out.

https://github.com/teddit-net/teddit

[2]: https://github.com/teddit-net/teddit