curl-impersonate VS instagram-private-api

> us[e] Akamai to block scraping

Would https://github.com/lwthiker/curl-impersonate help? Haven’t tried with Akamai, but did help with another widely used CDN that shall remain unnamed (but has successfully infused me with burning hate for their products after a couple of years’ worth of using an always-on VPN to bypass Internet censorship and/or a slightly unusual browser).

But before that, I had to create a development environment where I could do the coding. I used Docker and created a docker-compose.yml file on my local system to build a container. At first, I did that on an Arm based computer and the first problem appeared. Although RSS-Bridge was working fine, I couldn't get any data, and the reason was that Ko-Fi.com uses Cloudflare CDN. This is something that a lot of people had issues with in the past. RSS-Bridge solves that problem by using a special build of curl that can impersonate the four major browsers: Chrome, Edge, Safari & Firefox. But unfortunately, that library doesn't work well on Arm-based systems, so I had to move to my trusty Intel-based Linux computer.

Curl-Impersonate: https://github.com/lwthiker/curl-impersonate A special build of curl that can impersonate Chrome & Firefox

I haven’t seen a custom build of Wget, but for Curl there is curl-impersonate[1].

[1] https://github.com/lwthiker/curl-impersonate

If you're just trying to pull a file, the curl-impersonate could be a low-effort option.

This looks like an example. Never used it and the example does look like trash but with some playing you should find something useful. https://github.com/dilame/instagram-private-api/blob/23bf2b451dbc219eae6ed09fbc1a73a872528093/examples/account-followers.feed.example.ts

Simple, easy implementation of the Instagram private web API. Some API reference from jlobos/instagram-web-api Send DM using client from dilame/instagram-private-api

Nice work. What was your method to scrape a user's follower/following list? Did you use (instagram-private-api)[https://github.com/dilame/instagram-private-api]? AFAIK there's no official API for getting followers/following of a given user.

The most trivial check a website owner can do is checking the user-agent, which Python requests automatically sets to show its name, unless you configure your own. Trivial way to work around is to set your own user-agent to one that looks like a browser.

Specifically regarding Instagram, you can take a look at the implementation of https://github.com/dilame/instagram-private-api to understand more workarounds, as Instagram is getting better and better at working against the workarounds.