Our great sponsors
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
Hi, I am trying to pull a large amount of data via the PullShift reddit API (10's of millions of records) and I was wondering how an API rate limiter works? I am wondering this because, as of now, it will take A LONG time for me to pull all the data I need, even using something such as PMAW. So I am looking for solutions to speed up the process, mainly using multiple machines in parallel to pull different subsets of the data.
Hi, I am trying to pull a large amount of data via the PullShift reddit API (10's of millions of records) and I was wondering how an API rate limiter works? I am wondering this because, as of now, it will take A LONG time for me to pull all the data I need, even using something such as PMAW. So I am looking for solutions to speed up the process, mainly using multiple machines in parallel to pull different subsets of the data.