Our great sponsors
-
detect-headless
Access https://infosimples.github.io/detect-headless to run several headless detection tests against your browser.
-
-
WorkOS
The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
Do you know how to detect headless browser? For example, you can check your bot here: https://github.com/infosimples/detect-headless. Try to make your bot undetectable.
Other than that, have you looked into testing scrapers? Since scrapers are working with highly dynamic data writing good tests is quite a challange. For example, for parser monitoring using cerberus is a very cool tool which allows you to define loose requirements like "phone number should always be 9 numbers" etc.
Related posts
- Show HN: Config-file-validator – CLI tool to validate all your config files
- Do you think we need an open-source web scraping monitoring tool?
- Next REST Framework - Type-safe, self-documenting REST APIs for Next.js
- Providing ML team with data: normalized or denormalized?
- Show HN: Metashade – a Pythonic GPU shading/compute EDSL