google-play-scraper VS node-crawler

Also, you can use a complete third-party Google Play Store App scraping solution google-play-scraper. Third-party solutions are usually used to break the quota limit.

You need a scraper, like https://github.com/facundoolano/google-play-scraper

Hacker News is a good board to troll, they announce there a lot, also places like techcrunch and producthunt have saas companies featured. If you want a volume solution, https://github.com/facundoolano/google-play-scraper is good for scraping the play store, from which you can just search for what you want. I accumulated a database of 200K apps with that in a week.

web crawlers: these are applications that systematically map all of the links on websites, and create a searchable index. A popular JS tool here is node-crawler.

Before you write your own library for crawling, try some of the options out there. Many great Open Source libraries can achieve it: Scrapy, pyspider, node-crawler (Node.js), or Colly (Go). And many companies and services that provide you with scraping and crawling solutions.

We could write some snippet mixing all of these, but the best option in real life is to use a tool with it all like Scrapy, pyspider, node-crawler (Node.js), or Colly (Go). The idea being the snippets is to understand each problem on its own. But for large-scale, real-life projects, handling everything on our own would be too complicated.