Onboard AI learns any GitHub repo in minutes and lets you chat with it to locate functionality, understand different parts, and generate new code. Use it for free at www.getonboard.dev. Learn more →
Top 10 Ruby Scraper Projects
Create agents that monitor and act on your behalf. Your agents are standing by!Project mention: Ask HN: What is the correct way to deal with pipelines? | news.ycombinator.com | 2023-09-21
"correct" is a value judgement that depends on lots of different things. Only you can decide which tool is correct. Here are some ideas:
Your idea about a queue (in redis, or postgres, or sqlite, etc) is also totally valid. These off-the-shelf tools I listed probably wouldn't give you a huge advantage IMO.
Lightweight Ruby web crawler/scraper with an elegant DSL which extracts structured data from pages.
Learn any GitHub repo in 59 seconds. Onboard AI learns any GitHub repo in minutes and lets you chat with it to locate functionality, understand different parts, and generate new code. Use it for free at www.getonboard.dev.
A versatile Ruby web spidering library that can spider a site, multiple domains, certain links or infinitely. Spidr is designed to be fast and easy to use. (by postmodern)
Tanakai is a modern web scraping framework written in Ruby. A fork of Kimurai.Project mention: Tanakai: Modern web scraping framework written in Ruby | news.ycombinator.com | 2023-10-25
📰 Build RSS 2.0 feeds from websites (and JSON APIs) with a few CSS selectors.
🕸 Generates and delivers RSS feeds via HTTP. Docker image available! Create your own feeds or get started quickly with the included configs.Project mention: Ask HN: What RSS Reader do you use in 2022? | news.ycombinator.com | 2022-12-23
Collect and Analyze Billions of Data Points in Real Time. Manage all types of time series data in a single, purpose-built database. Run at any scale in any environment in the cloud, on-premises, or at the edge.
:jp::newspaper::mount_fuji: NHK News Web (Easy) word frequency (core list) scraper for Japanese language learners.
A simple rails scrapper app to count html tags of a web page.
A simple gem that crawls chans and retrieves visual content
Ruby Scraper related posts
Tanakai: Modern web scraping framework written in Ruby
1 project | news.ycombinator.com | 25 Oct 2023
Are you using Huginn? If so do you have any latest documentation?
1 project | /r/selfhosted | 15 Aug 2023
Generate RSS feed for any website using CSS selectors
2 projects | news.ycombinator.com | 14 Jul 2023
What web scrapers do you recommend.
1 project | /r/docker | 5 Jul 2023
Any recommendations for a open source replacement for If This Then That?
2 projects | /r/opensource | 1 Jul 2023
Looking for a web scrapper to detect changes to a webpage on a schedule
2 projects | /r/selfhosted | 27 Jun 2023
Where can I see Hokusai's Great Wave today?
1 project | news.ycombinator.com | 18 May 2023
A note from our sponsor - Onboard AI
getonboard.dev | 4 Dec 2023
What are some of the best open-source Scraper projects in Ruby? This list will help you: