I made a site that tracks the price of eggs at every US Walmart. The most expensive costs 3.4X more than the cheapest.

• xpath-scraper

  3 12 1.5 Rust

  Makes it simple to scrape websites with xpath structs.

It just came down to myself learning Rust several years ago. I actually used to code in Node.js and Java. Now, why I ended up sticking with Rust with this? Macros. An example of it being my scraper library. Instead of me having to redefine functions for each struct (class) that I want to apply XPATH evaluations for, the macros I made will do it for me. Proc Macros just make coding redundant things more straight-forward and easy to read. For example this is the example inside my library. This is what it (mostly) expands to once its' built. Imagine if you had to do that 20+ times. Also, it wraps an error handler around it too. Its' just more clean to work with.

• Watchmen

  1 1 10.0 Rust

  Watches for RSS/XML/XPATH Feed changes and XPATH Item(s) changes

Yes. Easily. I personally made my own program for more-easily watching for changes in a price and stock. I don't recommend using my program b/c it's weird to setup and I don't plan on updating it any time soon. You can see what I used it for in the pictures.

• InfluxDB

  www.influxdata.com featured

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• Newman

  244 6,718 7.5 JavaScript

  Newman is a command-line collection runner for Postman

  

[3B] Insomnia or Postman Useful for preparing the requests for Walmart. I recommend finding two different store which have different prices and saving what ids they are to test that you're doing everything correctly.

• RegExr

  579 9,541 0.0 JavaScript

  RegExr is a HTML/JS based tool for creating, testing, and learning about Regular Expressions.

[3A,2D] Regex Although you don't really need it for [2D] since in a programming language you can just replace text ("Now $") and ("$") with nothing ("") if you're not using a programming language its' useful so you can learn how to strip away specifics. Regex is used a lot in programming.

• insomnia

  225 33,067 9.7 JavaScript

  The open-source, cross-platform API client for GraphQL, REST, WebSockets, SSE and gRPC. With Cloud, Local and Git storage.

  

[3B] Insomnia or Postman Useful for preparing the requests for Walmart. I recommend finding two different store which have different prices and saving what ids they are to test that you're doing everything correctly.

• destreamer

  10 2,360 2.8 TypeScript

  Save Microsoft Stream videos for offline enjoyment.

I have used puppeteer (headless chromium) to scrape a link from a website in javascript, copying a part from the project of destreamer. Would you suggest/recommend another language way?

• reqwest

  57 9,130 8.9 Rust

  An easy and powerful Rust HTTP Client

In total for scraping: reqwest, tokio, and scraper-main. Those are I use to get the scraping started.

• SaaSHub

  www.saashub.com featured

  SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

  

• actix-web

  171 20,249 9.1 Rust

  Actix Web is a powerful, pragmatic, and extremely fast web framework for Rust.

  

To make a website I'd recommend Actix or Axum.

• TheReader

  3 30 7.2 Rust

  An organizer for you books (hopefully audio too) written in Rust Lang.

  

I would like to also note. I actually have a bad habit of doing everything in Rust to a fault. For example, I'm also working on a Book Reader and the Full Stack is in Rust. Even though I should've made the frontend in Typescript. I personally haven't touched up on Javascript or Java since I started learning Rust. I just love it too much.

Suggest a related project