I absolutely love web scraping.

This page summarizes the projects mentioned and recommended in the original post on /r/webscraping

Our great sponsors
  • SurveyJS - Open-Source JSON Form Builder to Create Dynamic Forms Right in Your App
  • InfluxDB - Power Real-Time Data Analytics at Scale
  • WorkOS - The modern identity platform for B2B SaaS
  • playwright_stealth

    playwright stealth

  • There is another plugin, https://github.com/AtuboDad/playwright_stealth but I never tried it.

  • curlconverter

    Transpile curl commands into Python, JavaScript and 27 other languages

  • Relevant tools: - Browser dev tools and front-end tooling to debug JS and reconstruct requests in your code - grep.app and SourceGraph to check open-source parsers for some URLs (often, there are such repositories) - curlconverter to quickly draft a script from the cURL command - Regex and regex playgrounds to extract data from inline JavaScript - GraphQL introspection tools - Optionally, Fiddler or Wireshark to intercept and debug network requests (I don't use but my teammate does)

  • SurveyJS

    Open-Source JSON Form Builder to Create Dynamic Forms Right in Your App. With SurveyJS form UI libraries, you can build and style forms in a fully-integrated drag & drop form builder, render them in your JS app, and store form submission data in any backend, inc. PHP, ASP.NET Core, and Node.js.

    SurveyJS logo
NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts