puppeteer-extra VS user.js

You could use https://github.com/berstend/puppeteer-extra/tree/master/packages/puppeteer-extra-plugin-stealth A plugin to escape anti bot detection

There is a good chance that the website is using Cloudflare to block web scrapers, which will require you to use a fortified headless browser to solve the JS challenges. Your options include the Puppeteer stealth plugin and Selenium undetected-chromedriver.

There are even Puppeteer plugins that will do it for you. [^1]

The best detection I've come across so far (i.e. before this release) has just required I run headless Chrome in headed mode. Granted, I don't do a ton of scraping -- mostly just pulling data out of websites so that I can play with it in aggregate using more civilized tools.

[1]: https://github.com/berstend/puppeteer-extra/tree/master/pack...

You didn't know? https://github.com/berstend/puppeteer-extra/wiki/Block-resources-without-request-interception

As for solutions, you are on point. Running a headless browser or using a web scraping API that does that for you (I work at one: https://scrapfly.io hi) is the easiest way to do it. Note that because of javascript fingerprinting you still need to fortify your headless browsers with various scripts like puppeteer-stealth.

Unfortunately, it's now trivial to bypass recaptcha: https://github.com/berstend/puppeteer-extra/tree/master/packages/puppeteer-extra-plugin-recaptcha

Use a fortified headless browser like the stealth plugin for puppeteer.

Re: firefox and privacy, if you want to use firefox for privacy, consider using https://github.com/arkenfox/user.js . There is a case to be made that Firefox (with arkenfox's user.js) is one of the best privacy-respecting but still fairly usable browsers.

For extensions, I recommend people follow the recommendations[1] in the arkenfox repo and either harden their firefox or use librewolf. Umatrix is unmaintained since 2019.

[1] https://github.com/arkenfox/user.js/wiki/4.1-Extensions

For macOS : Thunderbird and you can harden it even more with this : https://github.com/arkenfox/user.js

only arkenfox

This article is shit. https://github.com/arkenfox/user.js/ is what you want.

If you make time to dig through settings and change them away from their official use (99% of users don't), then you should use a customized setup (in this case, a user.js). That way, you're good to go no matter what Firefox fork you use.

> Firefox remains a stable option to come back to everytime

Don't get me wrong, I've been using Firefox for the last decade and I don't intend on using anything else for the foreseeable future, but Mozilla has no idea what they're doing with Firefox nowadays. Firefox View is the most useless thing I've ever seen, that expiring "independent voices" theme picker was some weird hippie stunt[1], the latest UI redesign which split the tab from the window looks hideous, and it's not like Firefox doesn't have things you can tweak for a more private experience[2]. I miss Firefox Test Pilot where they tried out different new features, I found a lot of them to be very useful but sadly lots of them didn't make it.

[1] https://blog.mozilla.org/en/products/firefox/firefox-news/in...

[2] https://github.com/arkenfox/user.js/

Like others have said you can customize the browser to the point that it doesn't even look like the default anymore. Or customize it to maximize privacy.