awesome-web-scraping
Proxyman
Our great sponsors
| awesome-web-scraping | Proxyman | |
|---|---|---|
| 6 | 43 | |
| 6,308 | 4,962 | |
| - | 3.6% | |
| 5.1 | 6.3 | |
| 17 days ago | 12 days ago | |
| Makefile | ||
| GNU General Public License v3.0 or later | - |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
awesome-web-scraping
-
Ask HN: LinkedIn sent me a cease and desist for my Chrome extension. Help?
>I can scrape linkedin with a python script. That doesn't mean linkedin can shut down python.
Well said!
Also, what about copy-and-paste? The last time I checked, data could be highlighted in the browser, copied, and pasted...
Does that mean that LinkedIn can shut down the copy-and-paste capability of your browser and/or operating system?
What about "Save Page As..." functionality (the ability of a browser to save a page offline?)
Can LinkedIn shut down "Save Page As..." ?
Also, what about the Print Screen (take a screen snapshot) capabilities of your operating system?
Can LinkedIn shut down that?
Finally, there's literally oodles of software that can be used for web scraping; what follows below is just one non-canonical list:
https://github.com/lorien/awesome-web-scraping
Is LinkedIn going to shut down all of that, at the same time?
Anyway, an excellent point about Python!
- Awesome-web-scraping – List of libraries, tools and APIs for web scraping
-
How does webscraping a website work and putting the data into my website?
Because at least for the scraping part there are open-source and paid services that will probably get you the data today if you need it (unless these are some really hard-to-scrape websites you're targeting) But if you are keen on learning yourself just scroll down this subreddit you will find many guides users shared along the years...
-
Russian Flag in Readme
E.g. how would a Ukrainian dev feel having his project showcased in this list, under the Russian flag?
[0] https://github.com/lorien/awesome-web-scraping/issues/136
- A central repository for scrapping scripts
Proxyman
- Ask HN: Black Friday Discounts/Specials
-
Show HN: Pākiki Proxy – An intercepting proxy for penetration pesting
I previously used Proxyman [1] on iOS to the http requests send over TLS. It worked rather nicely. Proxyman in this case starts a VPN which handles all the traffic. It uses custom certificate to decrypt the messages.
[1] https://proxyman.io/
-
A collection of useful Mac Apps
Proxyman - Price: Free (optional paid plans available) Modern and intuitive HTTP/HTTPS debugging proxy app for macOS.
-
What are your favorite apps that has active development? (frequent new features, bug fixes, etc)
I'm using self-developed app MindMac daily to talk with ChatGPT, Proxyman to capture network, TablePlus to access databases and CleanshotX to take screenshots. All of them are currently in an active status.
- Software Developer Mac Apps
-
Mac Power Users 690: Better Touch Tool with Andreas Hegenberg
Links and Show Notes:More Power Users: Ad-free episodes with regular bonus segmentsSubmit Feedbackfolivora.ai - Great Tools for your Mac!iPhone Praktikum 2009GitHub - quicklywilliam/multiclutch: Customization App for Macbooks with MultiTouch supportHopperFSMonitorProxyman · Native, Modern Web Debugging Proxy · Inspect network traffic from Mac, iOS, Android devices with easeCharles Web Debugging Proxy • HTTP Monitor / HTTP Proxy / HTTPS & SSL Proxy / Reverse ProxyBTT and Chat GPT
- PSA: Switching 2FA apps - Don't migrate, recreate
-
Ask HN: Side project of more that $2k monthly revenue what's your project?
I've never tried it. Maybe you should give it a try. If it doesn't work, please create a ticket at https://github.com/ProxymanApp/Proxyman/issues
I'm happy to support you.
-
Why is the iOS chess.com app trying to connect to Proxyman?
Basically, Atlantis is a framework from Proxyman to capture their HTTP/HTTPS traffic for debugging purposes. It only needs during development, and should be removed from the production (AppStore) build.
-
Software for monitoring data that is received through a Wi-Fi network?
Wonder if this application matches your demand: https://proxyman.io/
What are some alternatives?
proxy-list - A list of free, public, forward proxy servers. UPDATED DAILY!
grpc-tools - A suite of gRPC debugging tools. Like Fiddler/Charles but for gRPC.
awesome-micropython - A curated list of awesome MicroPython libraries, frameworks, software and resources.
flipper - A desktop debugging platform for mobile developers.
TabNine - AI Code Completions
mitmproxy - An interactive TLS-capable intercepting HTTP proxy for penetration testers and software developers.
Awesome-Warez - All your base are belong to us!
Requestly - 🚀 Most Popular developer tool for frontend developers & QAs to debug web and mobile applications. Redirect URL (Switch Environments), Modify Headers, Mock APIs, Modify Response, Insert Scripts & Record web sessions and share it with your teammates for debugging.
syntax-highlighter - Syntax Highlighter extension for Visual Studio Code (VSCode). Based on Tree-sitter.
atlantis - Capture HTTP/HTTPS, and Websocket from iOS app without proxy.
cookiecutter-poetry-pypackage - Cookiecutter template for poetry managed python package
httptoolkit - HTTP Toolkit is a beautiful & open-source tool for debugging, testing and building with HTTP(S) on Windows, Linux & Mac :tada: Open an issue here to give feedback or ask for help.