Our great sponsors
-
Spidey
A multi threaded web crawler library that is generic enough to allow different engines to be swapped in.
-
changedetection.io
The best and simplest free open source web page change detection, website watcher, restock monitor and notification service. Restock Monitor, change detection. Designed for simplicity - Simply monitor which websites had a text change for free. Free Open source web page change detection, Website defacement monitoring, Price change notification
-
WorkOS
The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
Didn't get a chance to reply earlier but depending on what you're trying to do, you might want a web crawler. I have a crawler on Github that I built for scraping in instances where someone doesn't have an API. If you go this route, I suggest doing it as a background task and go off cached data.
Something that comes to mind here, about the changing of the website. A deployment of https://github.com/dgtlmoon/changedetection.io might help to notice when the API has to be updated.
Related posts
- Tell me your most exotic selfhosted solution, the crazier, the better, no 0815 solutions!
- 0.45.8 is out! new release :)
- free alternative for changedetection.io
- Where to get short 'flashy product intro video' made? is there a service?
- changedetection.io releases version 0.45.3! (Whats new since our last Reddit update in 0.42!)