-
action-scheduler
A scalable, traceable job queue for background processing large queues of tasks in WordPress. Specifically designed for distribution in WordPress plugins (and themes) - no server access required.
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
I’m actually working on a plugin that does exactly this. I use a library called action scheduler to split the import into one “action” per post and then use this library to process those actions in the background using “asynchronous actions” for creating and updating posts. This makes it so that you don’t run into issues with timeouts or import process hogging massive amount of server resources.