Cloudproxy – hide your scrapers IP behind the cloud

• cloudproxy

  7 1,349 2.2 Python

  Hide your scrapers IP behind the cloud. Provision proxy servers across different cloud providers to improve your scraping success.

• FlareSolverr

  39 5,608 8.3 Python

  Proxy server to bypass Cloudflare protection

  

Even though I only do it for hobby projects, crawling pages is becoming increasingly difficult unless you are a big player like Google or Microsoft with a whitelisted IP range.

I've had some success in scraping lately with a similar project called FlareSolverr(1).

It's purpose it to get you access to sites which won't let you crawl unless you are using a real browser (e.g amazon, instagram). It doesn't hide your IP but uses puppeteer with stealth mode to get you access to otherwise restricted urls.

(1) https://github.com/FlareSolverr/FlareSolverr

• SurveyJS

  surveyjs.io sponsored

  Open-Source JSON Form Builder to Create Dynamic Forms Right in Your App. With SurveyJS form UI libraries, you can build and style forms in a fully-integrated drag & drop form builder, render them in your JS app, and store form submission data in any backend, inc. PHP, ASP.NET Core, and Node.js.

  

Suggest a related project