|15 days ago||8 days ago|
|MIT License||GNU General Public License v3.0 only|
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
A simple solution to rotate proxies or how to spin up your own rotation proxy server with Puppeteer and only a few lines of JS code
1 project | reddit.com/r/webscraping | 5 Mar 2021
I'm currently implementing concurrency conditions at project/proxy/domain/session level in https://github.com/get-set-fetch/scraper . On each level you can define the maximum number of requests and the delay between two consecutive requests.
Web scraping content into postgresql? Scheduling web scrapers into a pipeline with airflow?
1 project | reddit.com/r/webscraping | 15 Feb 2021
If you're familiar with nodejs give https://github.com/get-set-fetch/scraper a try. Scraped content can be stored in sqlite, mysql or postgresql. It also supports puppeteer, playwright, cheerio or jsdom for the actual content extraction. No scheduler though.
Web Scraping 101 with Python
5 projects | news.ycombinator.com | 10 Feb 2021
I'm using this exact strategy to scrape content directly from DOM using APIs like document.querySelectorAll. You can use the same code in both headless browser clients like Puppeteer or Playwright and DOM clients like cheerio or jsdom (assuming you have a wrapper over document API). Depending on the way a web page was fetched (opened in a browser tab or fetched via nodejs http/https requests), ExtractHtmlContentPlugin, ExtractUrlsPlugin use different DOM wrappers (native, cheerio, jsdom) to scrape the content.
What is your “I don't care if this succeeds” project?
42 projects | news.ycombinator.com | 1 Feb 2021
https://github.com/get-set-fetch/scraper - I've been working (intermittently :) ) on a nodejs or browser extension scraper for the last 3 years, see the other projects under the get-set-fetch umbrella. Putting a lot more effort lately as I really want to do those Alexa top 1 million analysis like top js libraries, certificate authorities and so on. A few weeks back I've posted on Show:HN as you can do basic/intermediate? scraping with it.
Not capable of handling 1 mil+ pages as it still limited to puppeteer or playwright. Working on adding cheerio/jsdom support right now.
Show HN: Plugin Based, Batteries Included, Web Scraper
1 project | news.ycombinator.com | 19 Jan 2021
Installed a vpn(Protonvpn) on my system.After i rebooted my pc I am unable to use my Interent i tried restarting the network manager .I am a noob ig its a firewall issue but i dont know how to fix that :(.Plz help
1 project | reddit.com/r/archlinux | 18 Oct 2021
You might be interested in using vopono to run just one application e.g. Firefox through the VPN.
vpn on pi with pihole/plex/deluge and sonarr
1 project | reddit.com/r/selfhosted | 15 Oct 2021
I think vopono is a good tool for that. I'm not really familiar setting up VPN on linux and kinda lazy, so I really liked vopono, it has some automatic configurations for some popular VPN providers but you can also do a custom one.
How do I use a VPN/Proxy only for yt-dlp
1 project | reddit.com/r/youtubedl | 20 Sep 2021
How to contribute to open source or Linux kernel?
6 projects | reddit.com/r/linux | 15 Sep 2021
As for writing FOSS applications in general, you need to find things that you'd like to work on. For example, I wrote vopono since I wanted to be able to run only Firefox through a VPN connection and easily swap it between countries. Now I'm working on contributing to the Rust netlink crate to hopefully make it as comprehensive as pyroute2 (or the respective libraries in C and Go).
1 project | news.ycombinator.com | 14 Sep 2021
Mullvad are the best VPN service out there by far IMO - and writing vopono: https://github.com/jamesmcm/vopono - I used many of them!
I think most VPN users just want something to access different Netflix catalogues though, and Mullvad doesn't play that cat and mouse game.
(New Discussion) What are you working on right now?
13 projects | reddit.com/r/archlinux | 24 Aug 2021
Eventually trying to port vopono to use syscalls / rtnetlink messages instead of spawning ip commands directly.
Can't install Mullvad.
1 project | reddit.com/r/archlinux | 2 Apr 2021
While it's not directly related to this issue, you might be interested in using vopono to be run only specific applications through it (or different servers) - I'm a fellow Mullvad user.
Police warn students to avoid science website
1 project | reddit.com/r/LabourUK | 20 Mar 2021
If you need to avoid it regardless of your ISP, I would recommend using Mullvad and vopono.
Is Sci-Hub blocked in your country? If so, how do you access it?
1 project | reddit.com/r/scihub | 3 Mar 2021
I use mullvad, specifically with vopono.
Disabling IPv6 for OpenVPN tunnels can speed up the VPN
1 project | reddit.com/r/linux | 2 Mar 2021
This is an option in vopono for example.
What are some alternatives?
playwright-python - Python version of the Playwright testing and automation library.
pyppeteer - Headless chrome/chromium automation library (unofficial port of puppeteer)
SDRPlusPlus - Cross-Platform SDR Software
mcpp - Minecraft server written in C++
Scrapy - Scrapy, a fast high-level web crawling & scraping framework for Python.
listudy - Listudy - chess training server
sxhkd - Simple X hotkey daemon
Arthur - How to build your own AI art installation from scratch [Moved to: https://github.com/maxvfischer/DIY-ai-art]
rssguard - RSS Guard is simple feed reader which supports RSS/ATOM/JSON and many web-based feed services.
manim - A community-maintained Python framework for creating mathematical animations.
commutative-algebra - An introduction to the basic ideas of commutative algebra
go-plugin - Golang plugin system over RPC.