I absolutely love web scraping.

This page summarizes the projects mentioned and recommended in the original post on reddit.com/r/webscraping

Our great sponsors
  • Appwrite - The Open Source Firebase alternative introduces iOS support
  • SonarQube - Static code analysis for 29 languages.
  • JetBrains - Developer Ecosystem Survey 2022
  • Scout APM - Less time debugging, more time building
  • playwright_stealth

    playwright stealth

    There is another plugin, https://github.com/AtuboDad/playwright_stealth but I never tried it.

  • curlconverter

    Generate code from cURL commands

    Relevant tools: - Browser dev tools and front-end tooling to debug JS and reconstruct requests in your code - grep.app and SourceGraph to check open-source parsers for some URLs (often, there are such repositories) - curlconverter to quickly draft a script from the cURL command - Regex and regex playgrounds to extract data from inline JavaScript - GraphQL introspection tools - Optionally, Fiddler or Wireshark to intercept and debug network requests (I don't use but my teammate does)

  • Appwrite

    Appwrite - The Open Source Firebase alternative introduces iOS support . Appwrite is an open source backend server that helps you build native iOS applications much faster with realtime APIs for authentication, databases, files storage, cloud functions and much more!

NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts