juice VS cheerio

Another tool worth mentioning is the Responsive Email CSS Inliner by HTML Email. Just like other conversion tools, this inliner works similarly by reading your embedded CSS in one window and making it inline in another. The feature that makes it stand out from the rest is that it also has a third window that demonstrates what the actual email would look like, both on mobile and desktop. Additionally, there is an option to download a .html file of the email. Since the tool is powered by Juice, an open-source project, it can also be easily integrated into your Node.js projects and works in client JavaScript.

Juice

And it in-lines tagged HTML wonderfully. Works well.

Still, it's overkill, brings with it dependencies cheerio, mensch and slick and a look into the code behind the on-line demo makes rapidly clear that Juice is parsing the style sheets and applying them in what looks to be, must be, a product of its age. It has over 1500 lines of Javascript and the repo was created in 2011 - the state of Javascript was woeful then compared with today (though not as woeful as in 2001 😏).

So we have to give Juice full credit and admiration for its age and endurance, and that it's still being maintained and is useful. But I'll respectfully pass it by as I'm looking for a minimalist, native Javascript solution as far as possible, using the state of Javascript in 2021 a whole decade later (which in IT terms is an eon).

inline-css 👎

Immediately disconcerting is the npm documentation for this package. It says simply "Inspired by juice" and fails to tell us why the authors felt that embarking on a new project was worth their while. It hints at the fact that it's better than Juice in some way or other (why else start a new project inspired by it?), but lays no claim as to if or how that might be the case. A dark horse.

The github repo dates to 2015 and so, it's not as long-in-the-tooth as Juice, but still, it likely leans on a more meager JavaScript standard than we have available today and quite possibly, like Juice resorts to parsing CSS code and interpreting it. It has about 1700 lines of Javascript and so is even larger than Juice! The last code contributions on github are early 2020 so at least it's being maintained which is good.

Still, it also depends on cheerio, and so falls a little short of a native solution but worse, it does not provide us with any suggestion as to run-context (server side or client side).

It's on npm, which is Node.js which suggests server-side JavaScript. It doesn't rule out that it works client-side, but there is zero indication provided that it does nor a test site, the way Juice provided. And like it or not, I associate npm primarily with Node.js and server side applications (rightly or wrongly).

So for my needs, it goes respectfully into the too-hard basket. Failed, in a sense, by the poor quality of its own introductory presentation. An irony, because there is some nice documentation there, just not a nice introduction that spells out whether it runs client-side or not, how to if so, and how it's different from Juice and why we'd use it. All the obvious questions I have surfing in on it.

css-inliner 👎

Long story short, the introductory documentation fails this package even more than that for inline-css did. Again, it's on npm, and so looks targeted at server-side use and makes no effort to mention the other two (Juice or inline-css), or differentiate itself from them. The github repo dates back to 2015 as well and so it looks at some level to be an example of modern parallel development, this and inline-css emerging at the same time. And unsurprisingly, it again has around 1600 lines of JavaScript. The last code contribution was 2020 so again, at least it's being maintained which is good I guess.

Still, no hint as to the run-context it supports (server-side and/or client-side) but hints, via its npm references and templating languages that it's a server-side tool not a client-side tool. Which doesn't rule out client-side application but again no claim that it works in that context, and no tips on how (if it does) and no demo site, so it's a respectful pass. Still seeking a lean native Javascript solution in preference to some pet project lacking clarity and exhibiting bloat.

Native Javascript 👍

The failure to find an existing solution, leads me to (seeking) a native JavaScript implementation and the journey of discovery that lies behind that.

document.styleSheets provides us with a collection of CSSStyleSheets and CSSStyleSheet.rules in turn provides all of the CSSStyleRules in use and Element.matches() tells us if CSSStyleRule.selectorText applies to a given element.

window.getComputedStyle() provides rapid access to the complete set of computed styles. That is, crucially it has taken all those styles that position things relative to the browser window and calculated where, in real coordinates it landed. Among other things. Crucially, it also has resolved all of CSS variables. It describes what an element actually looks like on the screen, rather than in the abstract CSS.

We can use the style sheets and the computed styles to update the style attribute of each element and then, when all is done the innerText and the innerHTML or outerHTML are available for the the text/plain and text/html MIME parts respectively.

Given we're in-lining styles, and we don't want to actually change the element in the DOM (we're just aiming to copy it to the clipboard in one form or another), we have to take a copy of the DOM element and in-line the styles on that. We have to use the computed style from the source element however as it is rendered on screen and the copy is not (hence has not got computed styles). To wit, we will have a source and target element one that is in the DOM and provides computed styles, and one that is not in the DOM and receives in-lined style attributes.

Importantly an implementation like this is maybe 200, or 300 lines of JavaScript and not 1500.

That more or less covers us and we have a terse native JavaScript in-lining solution!

But ... performance, performance ... turns out that it's not slow, but it's not fast either, let's take a look at performance next, before we dive into an actual code implementation of the above schema - next week's article.

Cheerio is your ticket to the world of server-side magic, allowing you to manipulate HTML and XML documents with jQuery-like syntax. It’s perfect for web scraping, data extraction, or just making sense of the mess that is web content. With Cheerio, you get to play around with the DOM, use CSS selectors, and basically do all the cool things you'd do in the browser, but server-side.

In this guide, we'll be extracting information from Amazon product pages using the power of TypeScript in combination with the Cheerio and Crawlee libraries. We'll explore how to retrieve and extract detailed product data such as titles, prices, image URLs, and more from Amazon's vast marketplace. We'll also discuss handling potential blocking issues that may arise during the scraping process.

Used libraries Axios - it is a promise HTTP clients to make requests to the specified URL. Cheerio- it is a library for parsing and manipulating HTML that is commonly used here for extracting data from downloaded HTML content. Apify SDK- it is for building Apify Actors, that is utilized for initializing actor environments, getting input data, and pushing extracted data to the dataset.

Nice. I've used Cheerio for this in the past: https://github.com/cheeriojs/cheerio?tab=readme-ov-file#sele...

For this article, I will be using the TypeScript Starter template as shown in the screenshot above. This comes with Nodejs, Cheerio, Axios

> I'm not sure why Python web scraping is so popular compared to Node.js web scraping

Take this with a grain of salt, since I am fully cognizant that I'm the outlier in most of these conversations, but Scrapy is A++ the no-kidding best framework for this activity that has been created thus far. So, if there was scrapyjs maybe I'd look into it, but there's not (that I'm aware of) so here we are. This conversation often comes up in any such "well, I just use requests & ..." conversation and if one is happy with main.py and a bunch of requests invocations, I'm glad for you, but I don't want to try and cobble together all the side-band stuff that Scrapy and its ecosystem provide for me in a reusable and predictable way

Also, often those conversations conflate the server side language with the "scrape using headed browser" language which happens to be the same one. So, if one is using cheerio <https://github.com/cheeriojs/cheerio> then sure node can be a fine thing - if the blog post is all "fire up puppeteer, what can go wrong?!" then there is the road to ruin of doing battle with all kinds of detection problems since it's kind of a browser but kind of not

I, under no circumstances, want the target site running their JS during my crawl runs. I fully accept responsibility for reproducing any XHR or auth or whatever to find the 3 URLs that I care about, without downloading every thumbnail and marketing JS and beacon and and and. I'm also cognizant that my traffic will thus stand out since it uniquely does not make the beacon and marketing calls, but my experience has been that I get the ban hammer less often with my target fetches than trying to pretend to be a browser with a human on the keyboard/mouse but is not

Web scraping is a powerful technique used to extract data from websites. In this tutorial, we'll explore how to perform web scraping using Node.js, Axios for making HTTP requests,Cheerio for parsing HTML content and also json2csv for converting json data to csv. We'll scrape product data from a sample website, "https://scrapeme.live/shop/".

Web scraping, while immensely useful, often requires developers to navigate a sea of tools and libraries, each with its own quirks and intricacies. Whether it's JSDOM, Cheerio, Playwright, or even just plain old vanilla JS in the DevTools console, moving between these platforms can be a challenge.

While looking for a way to implement capo.js in BigQuery to understand how pages in HTTP Archive are ordered, I came across the Cheerio library, which is a jQuery-like interface over an HTML parser.

Cheerio is a JavaScript tool for parsing HTML and XML in Node.js. It provides APIs for traversing and manipulating the DOM of a webpage.