proxy-list VS awesome-web-scraping

Fetch them from this Repo daily https://github.com/clarketm/proxy-list. I created a cron Job which Pulls the repository and then Updates my config.

surfshark doesn't provide proxies, and if they did it would be one domain. https://github.com/clarketm/proxy-list might be worth a try

There is a daily update list here https://github.com/clarketm/proxy-list/blob/master/proxy-list-raw.txt

def filterOut_workingProxies(proxies): #FILTER OUT WORKING PROXIES workingProxies = [] #for i in range(10): for i in range(len(proxies)): p = proxies[i] print(f"testing: {p}...") proxy = { 'http' : 'http://'+p, #ERK:::https://stackoverflow.com/a/61466680 'https' : 'https://'+p} try: resp = requests.get("https://www.google.com", proxies=proxy, timeout=3) print(resp.status_code) if resp.status_code == 200 or str(resp.status_code)[0]=='2': print("+++++ WROKING\n") workingProxies.append(proxy) except: pass return workingProxies [...] #DAILY UPDATING PROXY LINKS: #CONST urls_github = ["https://github.com/ShiftyTR/Proxy-List/blob/master/proxy.txt", "https://github.com/clarketm/proxy-list/blob/master/proxy-list-raw.txt"] url_fplNet = "https://free-proxy-list.net"

>I can scrape linkedin with a python script. That doesn't mean linkedin can shut down python.

Well said!

Also, what about copy-and-paste? The last time I checked, data could be highlighted in the browser, copied, and pasted...

Does that mean that LinkedIn can shut down the copy-and-paste capability of your browser and/or operating system?

What about "Save Page As..." functionality (the ability of a browser to save a page offline?)

Can LinkedIn shut down "Save Page As..." ?

Also, what about the Print Screen (take a screen snapshot) capabilities of your operating system?

Can LinkedIn shut down that?

Finally, there's literally oodles of software that can be used for web scraping; what follows below is just one non-canonical list:

https://github.com/lorien/awesome-web-scraping

Is LinkedIn going to shut down all of that, at the same time?

Anyway, an excellent point about Python!

Because at least for the scraping part there are open-source and paid services that will probably get you the data today if you need it (unless these are some really hard-to-scrape websites you're targeting) But if you are keen on learning yourself just scroll down this subreddit you will find many guides users shared along the years...

E.g. how would a Ukrainian dev feel having his project showcased in this list, under the Russian flag?

[0] https://github.com/lorien/awesome-web-scraping/issues/136