Web scraping with rails

• ferrum

  9 1,647 8.5 Ruby

  Headless Chrome Ruby API

  

I've used Ferrum for a couple small scripts in the past before.

• usaf-aircraft

  1 1 0.0 HTML

• WorkOS

  workos.com sponsored

  The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

  

• food-trucks

  1 2 0.0 Ruby

• Nokogiri

  20 6,105 9.4 C

  Nokogiri (鋸) makes it easy and painless to work with XML and HTML from Ruby.

  

If the page is rendered as html you can use Nokogiri. It has great support and is pretty easy to get started with too.

• kimuraframework

  5 999 0.0 Ruby

  Kimurai is a modern web scraping framework written in Ruby which works out of box with Headless Chromium/Firefox, PhantomJS, or simple HTTP requests and allows to scrape and interact with JavaScript rendered websites

I've worked with https://github.com/vifreefly/kimuraframework in the past which was delightful.

• football_api

  1 4 0.0 Ruby

  A Ruby interface to the https://www.api-football.com.

• flatfish

  1 0 0.0 Ruby
  

Nokogiri let’s you use css selectors, so it’s good for scraping a known site. I made this about 10 years ago, but it might give you some ideas https://github.com/Loudon-Company/flatfish

• InfluxDB

  www.influxdata.com sponsored

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

Suggest a related project