chompjs VS Tailwind CSS

Although I really like using Puppeteer when building projects that utilize web scraping in JavaScript / Node.js, I decided that this time I’d go with Python for my web scraping needs. Selenium would have been an obvious choice, and I’ve even used it previously in both Python and Java, however I decided to go with Beautiful Soup — its simplicity and speed (relative to Selenium) were what won me over. I was looking to build a lightweight scraper that I could deploy on a server and run a few times a day, and Beautiful Soup did that perfectly. Well, almost perfectly, there was one thing it couldn’t do: scrape content that was generated via JavaScript after the page loaded. This initially became a problem, as a few of the suppliers used JS to load their content. After countless Stack Overflow and Reddit threads, I finally discovered ChompJS, which solved this exact problem.

Lastly, Tailwind CSS is a utility-first CSS framework packed with classes like flex, pt-4, text-center, and rotate-90 that can be composed to build any design, directly in your markup.

Tailwind CSS

Sidekiq is already configured along with assets, tailwindsCSS.

Finally, for our front end, we’re going to be pairing Next.js with the great combination of TailwindCSS and shadcn/ui so we can focus on building the functionality of the app and let them handle making it look awesome!

You can use any frontend framework you want — react-based tooling, however, has a natural advantage as it models everything as a function of state, which can map 1:1 with the concept in Burr. In the demo app we use react, react-query, and tailwind, but we’ll be skipping over this largely (it is not central to the purpose of the post).

Tailwind CSS: A utility-first CSS framework for rapidly building custom designs.

First, you need to make sure that you have a working Tailwind CSS project…

Tailwind CSS

With better CSS approaches like TailwindCSS and Vanilla Extract (which we're heavily using) it's much easier to maintain the UI and make sure it doesn't change unexpectedly. No more conflicting CSS classes, much less CSS specificity issues and much less CSS code in general.