utls-light VS curl-impersonate

This kind of flexibility is a non-goal of crypto/tls. We have a TLS stack with one of the best security track records because we implement an opinionated subset of the specification, amongst other things. Moreover, fingerprint evasion is a cat-and-mouse game we can't sustain in the six months Go release cycle.

That doesn't mean I don't care! I was just talking with a friend about this the other day, and I suggested it should be possible to make a small, easily maintained patch that focuses on chasing the fingerprint of one well-known browser. He implemented https://github.com/hellais/utls-light in that spirit, which looks like a viable solution to me.

Anyway, I think matching TLS fingerprints to HTTP User-Agent strings is a valid abuse prevention technique. Rejecting any non-browser fingerprint is bad, and websites should get pushback for that, but I am skeptical that's something they can reliably do without breaking any time Chrome flips a field study. TLS is not _that_ rusted shut.

> us[e] Akamai to block scraping

Would https://github.com/lwthiker/curl-impersonate help? Haven’t tried with Akamai, but did help with another widely used CDN that shall remain unnamed (but has successfully infused me with burning hate for their products after a couple of years’ worth of using an always-on VPN to bypass Internet censorship and/or a slightly unusual browser).

But before that, I had to create a development environment where I could do the coding. I used Docker and created a docker-compose.yml file on my local system to build a container. At first, I did that on an Arm based computer and the first problem appeared. Although RSS-Bridge was working fine, I couldn't get any data, and the reason was that Ko-Fi.com uses Cloudflare CDN. This is something that a lot of people had issues with in the past. RSS-Bridge solves that problem by using a special build of curl that can impersonate the four major browsers: Chrome, Edge, Safari & Firefox. But unfortunately, that library doesn't work well on Arm-based systems, so I had to move to my trusty Intel-based Linux computer.

Curl-Impersonate: https://github.com/lwthiker/curl-impersonate A special build of curl that can impersonate Chrome & Firefox

I haven’t seen a custom build of Wget, but for Curl there is curl-impersonate[1].

[1] https://github.com/lwthiker/curl-impersonate

If you're just trying to pull a file, the curl-impersonate could be a low-effort option.