How to use rotating proxies with Puppeteer

• pyppeteer

  1 3,410 0.2 Python

  Discontinued Headless chrome/chromium automation library (unofficial port of puppeteer) (by miyakogi)

Puppeteer is a high-level API to control headless Chrome. Most things that you can do manually in the browser can be done using Puppeteer, so it quickly became one of the most popular web scraping tool in Node.js and Python. Many developers use it for single-page applications (SPA) data extraction as it allows executing client-side Javascript. In this article, we are going to show how to set up a proxy in Puppeteer and how to spin up your own rotating proxy server.

• proxy-chain

  2 784 4.1 JavaScript

  Node.js implementation of a proxy server (think Squid) with support for SSL, authentication and upstream proxy chaining.

  

To spin up proxy rotation server you may use the handy library proxy-chain and ScrapingAnt free proxies list:

• InfluxDB

  www.influxdata.com sponsored

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

Suggest a related project