Is Selenium still a good choice?

• playwright-pool

  3 11 0.0 Python

  Demonstration on how to use async python to control multiple playwright browsers for web-scraping

But to summarize it - puppeteer and playwright are superior to Selenium. Mostly because they both have modern, async APIs. When it comes to API itself Playwright is a great choice, though it comes with a lot of default cruft (browser parameters etc) that make scrapers easier to identify. Async support is really important too as there's a lot of IO blocking in browser automation. With async API you can launch multiple asynchronous browser tabs and do something in one while the other is loading - which drastically speeds up web scraping. I published a short demo on github to illustrate this: playwright-pool if you want to learn more about async.

• puppeteer-extra

  28 6,056 0.0 JavaScript

  💯 Teach puppeteer new tricks through plugins.

That being said, if you're a beginner Selenium is a much more mature package so it has significantly more resources on StackOverflow and whatnot and Puppeteer has bigger community for avoiding web scraper detection (plugins like puppeteer-extra-plugin-stealth)

• InfluxDB

  www.influxdata.com sponsored

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• scrapy-playwright

  11 835 7.8 Python

  🎭 Playwright integration for Scrapy

  

This concern should be lifted if you are a Scrapy lover. There is a Scrapy integration for playwright, that gives you a lot of freedom and lets you operate from a Scrapy spider.

Suggest a related project