Show HN: I scraped 25M Shopify products to build a search engine

1. Geziyor

   1 2 2,710 1.1 Go

   Geziyor, blazing fast web crawling & scraping framework for Go. Supports JS rendering.

   

   As someone who has scraped millions of items myself, I had success using Geziyor (https://github.com/geziyor/geziyor) built in Go. Shopify sites are especially easy to scrape because they tend to share the same product data formatting and don't hide it behind JS rendering.

2. InfluxDB

   www.influxdata.com featured

   InfluxDB – Built for High-Performance Time Series Workloads. InfluxDB 3 OSS is now GA. Transform, enrich, and act on time series data directly in the database. Automate critical tasks and eliminate the need to move data externally. Download now.

   

3. showcase-songs-search

   2 3 182 5.0 JavaScript

   A site to instantly search 32M songs from the MusicBrainz songs database, using Typesense Search (an open source alternative to Algolia / ElasticSearch) ⚡ 🎵 🔍

   

   I'm biased, but I'd recommend exploring Typesense for search.

   It's an open source alternative to Algolia + Pinecone and e-commerce is a very common use-case.

   Here's a live demo with 32M songs: https://songs-search.typesense.org/

   Disclaimer: I work on Typesense.

4. usearch

   3 28 2,706 9.2 C++

   Fast Open-Source Search & Clustering engine × for Vectors & 🔜 Strings × in C++, C, Python, JavaScript, Rust, Java, Objective-C, Swift, C#, GoLang, and Wolfram 🔍

   

   As you scale, you may benefit from these two projects I maintain, and the Big Tech uses :)

   https://github.com/unum-cloud/usearch - for faster search

   https://github.com/unum-cloud/uform - for cheaper multi-lingual multi-modal embeddings

5. uform

   4 9 1,132 7.8 Python

   Pocket-Sized Multimodal AI for content understanding and generation across multilingual texts, images, and 🔜 video, up to 5x faster than OpenAI CLIP and LLaVA 🖼️ & 🖋️

   

   As you scale, you may benefit from these two projects I maintain, and the Big Tech uses :)

   https://github.com/unum-cloud/usearch - for faster search

   https://github.com/unum-cloud/uform - for cheaper multi-lingual multi-modal embeddings

Suggest a related project