URS VS PRAW

If they don't want you to use their API just respect their wishes and scrape Reddit. https://github.com/JosephLai241/URS it's the only moral thing we can do.

For future visitors the steps that this user needed were: (Install python 3.7) git clone --depth=1 https://github.com/JosephLai241/URS.git cd URS pip3 install . -r requirements.txt Then edit the .env contained in the directory with the reddit api credentials

Just used this repo to do exactly this for a PhD program project. https://github.com/JosephLai241/URS

Shameless plug - I wrote URS which allows you to export comments to JSON in a thread structure. Take a look at the samples branch for examples.

I'm new to python, and coding generally. I'm using a great tool called the Universal Reddit Scraper (https://github.com/JosephLai241/URS) to pull some reddit data. It allows you to scrape subreddits, among other things. It creates a CSV file with list of submissions in a given subreddit with each one's ID as a column.

Because I'm not sure this guy does everything you want https://github.com/JosephLai241/URS

Toteutettu Universal Reddit Scraperillä.

I'm... not sure I understand the question? PRAW still works just fine for "personal use" of the reddit API.

Thank you for pointing out. So there is no need then for the access token? Only with the refresh token is enough? To be honest I took a look at it but I did not expect that to be under authentication as strictly speaking, the user already made the authentication. Also I took a look at the code at https://github.com/praw-dev/praw/blob/master/praw/reddit.py and I did not get a hint whether was possible to pass it or not. I am just saying this to let you know I tried to search for the answer before asking. Again thank you for the help.

To get the relevant IDs, you can use something like PRAW to query the subreddit for the top 1000 posts for example.

Huge thanks to the contributors to PRAW, which is the Python package that does all the heavy lifting relating to Reddit's API that I need for this script.

However, in practice duplicate items were yielded with these smaller numbers. So I increased the limit briefly to 250 in October 2016, and then increased it finally to 301 in December 2016 in order to resolve https://github.com/praw-dev/praw/issues/673. That issue provides an explanation for how 301 came to be.

Why? You gotta be ready for any status code. Even 777.