Our great sponsors
-
SurveyJS
Open-Source JSON Form Builder to Create Dynamic Forms Right in Your App. With SurveyJS form UI libraries, you can build and style forms in a fully-integrated drag & drop form builder, render them in your JS app, and store form submission data in any backend, inc. PHP, ASP.NET Core, and Node.js.
-
openapi-generator
OpenAPI Generator allows generation of API client libraries (SDK generation), server stubs, documentation and configuration automatically given an OpenAPI Spec (v2, v3)
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
That's why there's https://matrix.org/
I wrote a npm package which can scrape the data some time ago, here it is.
Why scrapy if you have teddit? These kinds of projects are the best tbh
Do you mean yt-dlp?
And lastly, consider your target. For example, I used to have a little scrapping tool for StandardEbooks. They're a small, little-known project. I have no idea how their stack looks like, but I assume they didn't have supercomputers running that site, at least back in the day. These guys do lots of thankless job to give away quality products for free. So you're damned right I checked their robots.txt before doing it (delightful, by the way), and limited that scrapper to one request at a time. Even put a waiter between dowloads, just to be extra nice. And not like I will ever download hundreds of books at a time (I mostly used it to automate downloading the EPUB and KEPUB version for my Kobo for one book; yes, several hours of work to save me a click...), but I promised myself I would never do massive bulk downloads, as that's a bennefit for their paying Patrons.
https://github.com/loftongentry/loftongentry Here ya go. I haven’t updated the README, as it’s not done, but that’s it’s current state so far
Related posts
- I'm a little embarrassed to ask but how do you actually consume an API like those built with FastAPI?
- Element: A sovereign and secure communications platform
- Bypass CORS errors while testing your APIs using Hoppscotch 🔧
- Discord is nuking Nintendo Switch emulator devs and their entire servers
- Google fights Invidious (a privacy YouTube Front end)