Top 10 Ruby Scraper Projects

Huginn

121 41,441 7.2 Ruby

Create agents that monitor and act on your behalf. Your agents are standing by!

Project mention: Create agents that monitor and act on your behalf | news.ycombinator.com | 2024-03-24

Wombat

1 1,303 2.2 Ruby

Lightweight Ruby web crawler/scraper with an elegant DSL which extracts structured data from pages.
WorkOS

workos.com sponsored

The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
kimuraframework

5 999 0.0 Ruby

Kimurai is a modern web scraping framework written in Ruby which works out of box with Headless Chromium/Firefox, PhantomJS, or simple HTTP requests and allows to scrape and interact with JavaScript rendered websites
spidr

0 792 6.4 Ruby

A versatile Ruby web spidering library that can spider a site, multiple domains, certain links or infinitely. Spidr is designed to be fast and easy to use. (by postmodern)
tanakai

3 260 6.1 Ruby

Tanakai is a modern web scraping framework written in Ruby. A fork of Kimurai.

Project mention: Tanakai: Modern web scraping framework written in Ruby | news.ycombinator.com | 2023-10-25

html2rss

0 111 4.9 Ruby

📰 Build RSS 2.0 feeds from websites (and JSON APIs) with a few CSS selectors.
html2rss-web

1 80 8.0 Ruby

🕸 Generates and delivers RSS feeds via HTTP. Docker image available! Create your own feeds or get started quickly with the included configs.
InfluxDB

www.influxdata.com sponsored

Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
nhkore

1 13 1.8 Ruby

:jp::newspaper::mount_fuji: NHK News Web (Easy) word frequency (core list) scraper for Japanese language learners.
rails-urltohtml

1 5 4.9 Ruby

A simple rails scrapper app to count html tags of a web page.
chanCrawler

1 4 0.0 Ruby

A simple gem that crawls chans and retrieves visual content

NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020).