Our great sponsors
-
-
WorkOS
The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
Another option available to you is to run with an intercepting proxy between Scrapy and the target site, which similarly will capture all the content, and has the advantage of already being in a database. It may even offer the checksumming that I mentioned for free
NOTE:
The number of mentions on this list indicates mentions on common posts plus user suggested alternatives.
Hence, a higher number means a more popular project.