spidr

A versatile Ruby web spidering library that can spider a site, multiple domains, certain links or infinitely. Spidr is designed to be fast and easy to use. (by postmodern)

Spidr Alternatives

Similar projects and alternatives to spidr

  • Mechanize

    Mechanize is a ruby library that makes automated web interaction easy.

  • MetaInspector

    Ruby gem for web scraping purposes. It scrapes a given URL, and returns you its title, meta description, meta keywords, links, images...

  • InfluxDB

    Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

    InfluxDB logo
  • spidy Web Crawler

    The simple, easy to use command line web crawler.

  • instabot.rb

    An instagram bot works without instagram api, only needs your username and password. written in ruby

  • crawlee

    29 spidr VS crawlee

    Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

  • Wombat

    1 spidr VS Wombat

    Lightweight Ruby web crawler/scraper with an elegant DSL which extracts structured data from pages.

  • WorkOS

    The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

    WorkOS logo
  • PSpider

    0 spidr VS PSpider

    简单易用的Python爬虫框架,QQ交流群:597510560

  • Upton

    0 spidr VS Upton

    A batteries-included framework for easy web-scraping. Just add CSS! (Or do more.)

  • The Hawker Ruby gem

    The Hawker gem is a web scraper which allows you to pull the basic information for given social media profile URL

NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a better spidr alternative or higher similarity.

spidr reviews and mentions

Posts with mentions or reviews of spidr. We have used some of these posts to build our list of alternatives and similar projects.

We haven't tracked posts mentioning spidr yet.
Tracking mentions began in Dec 2020.

Stats

Basic spidr repo stats
0
792
6.4
3 months ago

postmodern/spidr is an open source project licensed under MIT License which is an OSI approved license.

The primary programming language of spidr is Ruby.


Sponsored
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com