Ruby Web Crawling

Open-source Ruby projects categorized as Web Crawling
Edit details
Topics: Ruby Scraper Crawler web-scraping Web
Power Real-Time Data Analytics at Scale
Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
www.influxdata.com
Sponsored

Top 16 Ruby Web Crawling Projects

  • Mechanize

    Mechanize is a ruby library that makes automated web interaction easy.

  • anemone

    Anemone web-spider framework

  • InfluxDB

    Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

    InfluxDB logo

  • FastImage

    FastImage finds the size or type of an image given its uri by fetching as little as needed

  • Wombat

    Lightweight Ruby web crawler/scraper with an elegant DSL which extracts structured data from pages.

  • MetaInspector

    Ruby gem for web scraping purposes. It scrapes a given URL, and returns you its title, meta description, meta keywords, links, images...

  • spidr

    A versatile Ruby web spidering library that can spider a site, multiple domains, certain links or infinitely. Spidr is designed to be fast and easy to use. (by postmodern)

  • pismo

    Extracts machine-readable metadata and content from Web pages

  • WorkOS

    The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

    WorkOS logo

  • vessel

    Fast high-level web crawling Ruby framework (by rubycdp)

  • LinkThumbnailer

    Ruby gem that fetches images and metadata from a given URL. Much like popular social website with link preview.

  • tanakai

    Tanakai is a modern web scraping framework written in Ruby. A fork of Kimurai.

    • Project mention: Tanakai: Modern web scraping framework written in Ruby | news.ycombinator.com | 2023-10-25

  • instabot.rb

    An instagram bot works without instagram api, only needs your username and password. written in ruby

  • clauneck

    A tool for scraping emails, social media accounts, and much more information from websites using Google Search Results.

    • Project mention: Clauneck: A command line tool and a ruby gem for scraping emails, social media accounts, and much more information from websites using Google Search Results. | /r/bigdata | 2023-07-11

  • The Hawker Ruby gem

    The Hawker gem is a web scraper which allows you to pull the basic information for given social media profile URL

  • google-search-results-ruby

    Google Search Results via SERP API Ruby Gem

  • Supplejack API

    Supplejack API Mountable Engine

  • socials_regex

    🪡 Social account detection and extraction in ruby, e.g. for crawling/scraping.

    • Project mention: socials_regex new gem for Social account detection and extraction in ruby, e.g. for crawling/scraping. Detect and extract URLs of social accounts: throw in URLs, get back URLs of social media profiles by type. | /r/ruby | 2023-07-02

    Github: socials_regex

  • SaaSHub

    SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

    SaaSHub logo
NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020).

Ruby Web Crawling related posts

Index

What are some of the best open-source Web Crawling projects in Ruby? This list will help you:

Project Stars
1 Mechanize 4,353
2 anemone 1,613
3 FastImage 1,355
4 Wombat 1,303
5 MetaInspector 1,021
6 spidr 792
7 pismo 746
8 vessel 602
9 LinkThumbnailer 510
10 tanakai 260
11 instabot.rb 153
12 clauneck 140
13 The Hawker Ruby gem 70
14 google-search-results-ruby 48
15 Supplejack API 17
16 socials_regex 8

Sponsored
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com