InfoSpider VS undetected-chromedriver

kangvcar/InfoSpider (Python): INFO-SPIDER 是一个集众多数据源于一身的爬虫工具箱🧰，旨在安全快捷的帮助用户拿回自己的数据，工具代码开源，流程透明。支持数据源包括GitHub、QQ邮箱、网易邮箱、阿里邮箱、新浪邮箱、Hotmail邮箱、Outlook邮箱、京东、淘宝、支付宝、中国移动、中国联通、中国电信、知乎、哔哩哔哩、网易云音乐、QQ好友、QQ群、生成朋友圈相册、浏览器浏览历史、12306、博客园、CSDN博客、开源中国博客、简书。

This command-line tool clicks ads for a certain query on Google/Bing search using undetected_chromedriver package. Supports proxy, running multiple simultaneous browsers, ad targeting/exclusion, and running in loop.

Ultrafunkamsterdam created a ChromeDriver that has excellent support for bypassing all major bot detection systems such as Distil, Datadome, Cloudflare, and others.

Undetected_ChromeDriver also works on Brave Browser and many other Chromium-based browsers. For more, you can check out this project on GitHub.

Thanks a lot for your help. You were very passionate about it which means you cared. My mom called all my aunts and cousins gushing that I have written a project in 6 languages, I mostly look for people to fix the mistakes I have made even if the entire thing is a mistake. I think you and another person on C subreddit are the only person who went through this code. The person on that sub marked a mistake I have made regarding C and you marked the problem with everything which means I have gotten enough out of this. I've seen stupid people release bullshit "libraries" and stupider people using them and never know that they are in the wrong because they take every criticism as an insult. I told this genius that not only his conduct is extremely unprofessional but that he's misleading everyone because the usernames people use to access those sites will eventually get banned and he will have thousands of dollars people paid for botted users on his conscious. But he cringefully ignored me. He's such a dummy he's added a freaking shebang to files that are meant to be ran as a library! Jesus. Sorry I'm meandering.

There is a good chance that the website is using Cloudflare to block web scrapers, which will require you to use a fortified headless browser to solve the JS challenges. Your options include the Puppeteer stealth plugin and Selenium undetected-chromedriver.

I have used this project before. I think the author probably can create something lightweight with his knowledge. For my purposes I couldn’t find a Cloudflare bypass script that worked so I had to use this instead.

https://github.com/ultrafunkamsterdam/undetected-chromedrive...

Modded chromedriver + rotating user string + some reliable proxy should knock simple systems. Some anti-scraping systems are quite advanced, however.

Fortified Headless Browser: Depending on the anti-bot protection the website is using you make need to use a fortified headless browser that can solve its JS challenges without giving its identity away. Your options include the Puppeteer stealth plugin and Selenium undetected-chromedriver.