-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
-
-
Although I really like using Puppeteer when building projects that utilize web scraping in JavaScript / Node.js, I decided that this time I’d go with Python for my web scraping needs. Selenium would have been an obvious choice, and I’ve even used it previously in both Python and Java, however I decided to go with Beautiful Soup — its simplicity and speed (relative to Selenium) were what won me over. I was looking to build a lightweight scraper that I could deploy on a server and run a few times a day, and Beautiful Soup did that perfectly. Well, almost perfectly, there was one thing it couldn’t do: scrape content that was generated via JavaScript after the page loaded. This initially became a problem, as a few of the suppliers used JS to load their content. After countless Stack Overflow and Reddit threads, I finally discovered ChompJS, which solved this exact problem.
If you’re into programming, you can check out the repo here. If you’re in Israel and looking for some good beer, you can check out IsraBrew here.
Once scraped, the beers were then stored in an in-memory SQLite database. Last time I had worked with a Flask backend I wrote out all the SQL commands inside of my Python code, but this time I decided to use the SQLAlchemy ORM package, which allowed me to do CRUD operations in Python without any need for raw SQL.
A simple Frontend built with my usual stack: React, TypeScript and TailwindCSS. I discovered this NPM package which made implementing different tabs for different suppliers super easy. You all know about React, I’m not going to go into much detail here. What I will say is — if you’re using React but not using Tailwind to style your components, you should be. I still use old school CSS (or SASS) for specific things, but for me Tailwind has become almost as invaluable as React — If I’m building a React project, I’ll be styling it with TailwindCSS.