curl-impersonate VS undetected-chromedriver

> us[e] Akamai to block scraping

Would https://github.com/lwthiker/curl-impersonate help? Haven’t tried with Akamai, but did help with another widely used CDN that shall remain unnamed (but has successfully infused me with burning hate for their products after a couple of years’ worth of using an always-on VPN to bypass Internet censorship and/or a slightly unusual browser).

But before that, I had to create a development environment where I could do the coding. I used Docker and created a docker-compose.yml file on my local system to build a container. At first, I did that on an Arm based computer and the first problem appeared. Although RSS-Bridge was working fine, I couldn't get any data, and the reason was that Ko-Fi.com uses Cloudflare CDN. This is something that a lot of people had issues with in the past. RSS-Bridge solves that problem by using a special build of curl that can impersonate the four major browsers: Chrome, Edge, Safari & Firefox. But unfortunately, that library doesn't work well on Arm-based systems, so I had to move to my trusty Intel-based Linux computer.

Curl-Impersonate: https://github.com/lwthiker/curl-impersonate A special build of curl that can impersonate Chrome & Firefox

I haven’t seen a custom build of Wget, but for Curl there is curl-impersonate[1].

[1] https://github.com/lwthiker/curl-impersonate

If you're just trying to pull a file, the curl-impersonate could be a low-effort option.

This command-line tool clicks ads for a certain query on Google/Bing search using undetected_chromedriver package. Supports proxy, running multiple simultaneous browsers, ad targeting/exclusion, and running in loop.

You can check this out. https://github.com/ultrafunkamsterdam/undetected-chromedriver If i didn't understand you wrong then this is what you're asking for.

403 often means that the server recognized the scraper and blocked you. If you use Selenium, this plugin is very good for passing bot detection: https://github.com/ultrafunkamsterdam/undetected-chromedriver.

Ultrafunkamsterdam created a ChromeDriver that has excellent support for bypassing all major bot detection systems such as Distil, Datadome, Cloudflare, and others.

One solution would be to install Selenium and then scrape using a real browser like Chrome. If this solution gets blocked, you could install obfuscation plugins like this very good one: https://github.com/ultrafunkamsterdam/undetected-chromedriver

Undetected_ChromeDriver also works on Brave Browser and many other Chromium-based browsers. For more, you can check out this project on GitHub.