jsdom VS simplecrawler

If you happen to be using React Testing Library in your project, you'll need to keep the jsdom dev dependency installed.

Readability.js requires a DOM object to parse the readable content from a website. That's why we create a DOM object with JSDOM and provide the HTML from the page and its current URL. By the way, the browser may have had to follow HTTP redirects, so the current URL doesn't necessarily have to be the one we provided initially. The parse function of the library returns the following result:

This was the basic usage of JSDOM. If you want to learn more, please check out this documentation.

Web scraping, while immensely useful, often requires developers to navigate a sea of tools and libraries, each with its own quirks and intricacies. Whether it's JSDOM, Cheerio, Playwright, or even just plain old vanilla JS in the DevTools console, moving between these platforms can be a challenge.

React testing library uses jsdom (or others like happy-dom) under the hood to emulate the browser environment and enable testing of different properties. but it doesn't render the component in an actual browser meaning the styles can't be computed and the returned computed style is just an object with the correct keys but empty strings as values.

Note: In order to use the library jsdom to scrape the HTML response we need to avoid the @import statements in the CSS.

The code uses jSDom to render the app but I was wondering if it's possible to run the app without the UI to be able to use a library like that to generate the string from react element.

JSDOM as the DOM environment for running our tests

We will use a module called Simplecrawler to scan the dev server. Then, the files will be stored in a variable. Since our module is a virtua module, when the user tries to import the URLs, then we return the sitemap.