Web Scraping in Python: Avoid Detection Like a Ninja

This page summarizes the projects mentioned and recommended in the original post on dev.to

Our great sponsors
  • WorkOS - The modern identity platform for B2B SaaS
  • InfluxDB - Power Real-Time Data Analytics at Scale
  • SaaSHub - Software Alternatives and Reviews
  • puppeteer

    Node.js API for Chrome

    Selenium, Puppeteer, and Playwright are the most used and known libraries. Avoiding them for performance reasons would be preferable, and they'll make scraping slower. But sometimes, there is no alternative.

  • colly

    Elegant Scraper and Crawler Framework for Golang

    We could write some snippets mixing all these, but the best option in real life is to use a tool with it all, like Scrapy, pyspider, node-crawler (Node.js), or Colly (Go).

  • WorkOS

    The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts