Scraping Using Python in Step by step approach

This page summarizes the projects mentioned and recommended in the original post on

Our great sponsors
  • Onboard AI - ChatGPT with full context of any GitHub repo.
  • WorkOS - The modern API for authentication & user identity.
  • InfluxDB - Power Real-Time Data Analytics at Scale
  • Playwright

    Playwright is a framework for Web Testing and Automation. It allows testing Chromium, Firefox and WebKit with a single API.

    Websites tend to protect their data and access. Defensive systems could take many possible actions. Stay with us to learn how to mitigate their impact or directly bypass bot detection using Requests or Playwright.

  • puppeteer

    Node.js API for Chrome

    Selenium, Puppeteer, and Playwright are the most used and known libraries. Avoiding them for performance reasons would be preferable, and they’ll make scraping slower. But sometimes, there is no alternative.

  • Onboard AI

    ChatGPT with full context of any GitHub repo. Onboard AI learns any GitHub repo in minutes and lets you chat with it to locate functionality, understand different parts, and generate new code. Use it for free at

NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts