Rcrawler
An R web crawler and scraper (by salimk)
RedditExtractor
A minimalistic R wrapper for the Reddit API (by ivan-rivera)
Our great sponsors
| Rcrawler | RedditExtractor | |
|---|---|---|
| 2 | 5 | |
| 344 | 82 | |
| - | - | |
| 0.0 | 3.3 | |
| about 2 years ago | 8 months ago | |
| R | R | |
| GNU General Public License v3.0 or later | GNU General Public License v3.0 only |
The number of mentions indicates the total number of mentions that we've tracked plus the number of user suggested alternatives.
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
Rcrawler
Posts with mentions or reviews of Rcrawler.
We have used some of these posts to build our list of alternatives
and similar projects.
-
Can R do recursive web crawling?
According to their GitHub page, it should be able to:
-
increasing scraping speed
I'm not an R expert, but here are a few links about concurrent web scraping in R: RCurl, rvest + furrr, RCrawler.
RedditExtractor
Posts with mentions or reviews of RedditExtractor.
We have used some of these posts to build our list of alternatives
and similar projects. The last one was on 2022-09-08.
-
Will RedditExtractoR be impacted by API changes?
IIRC RedditExtractor doesn't use OAuth2 so I think the 10reqs/min ratelimit will be applied to the library/client.
-
bulk subreddit datasets?
Sounds like this package might help you reach your objective. The timeframe you can capture will depend on the amount of activity within the subreddit of interest.
- Has anyone here used the Reddit API before (in R)?
-
Using RedditExtractoR to scrape flairs?
My apologies if the Reddit API flair is inappropriate here - RedditExtractoR does use the Reddit API, but it's technically distinct as a simplified package for R (see: https://github.com/ivan-rivera/RedditExtractor)
-
H3 Podcast YouTube Views Analysis
Great idea, yeah Reddit has an API too, and it looks like there are R & Python packages to access it - https://github.com/ivan-rivera/RedditExtractor
What are some alternatives?
When comparing Rcrawler and RedditExtractor you can also consider the following projects:
r-web-scraping-cheat-sheet - Guide, reference and cheatsheet on web scraping using rvest, httr and Rselenium.
Pushshift API - Pushshift API
scrapingant-client-python - ScrapingAnt API client for Python.
police-settlements - A FiveThirtyEight/The Marshall Project effort to collect comprehensive data on police misconduct settlements from 2010-19.
crypto - Cryptocurrency Historical Market Data R Package
reddit-awards-data - Dataset and visualizations of the most popular Reddit Awards, using the PRAW API.
imdb - Web-scraping and data visualization of IMDb's most popular movies in 2018
tuber - :sweet_potato: Access YouTube from R
polite - Be nice on the web