WKHTMLToPDF VS puppeteer

Some of those problems I had in the past, but at the time I was just rendering tables for a financial report. The main problem I remember having was the CSS part and the long generation time. Because I was not implementing the styling at the time, the CSS part was not really my problem, and I am sure wicked_pdf provides some default styles to help in this part. The long processing times were a problem because we were generating pdfs with over 100 pages, this process would take about 5 min and would get worse if more pdfs were being requested in parallel. I can’t remember what the solution was at the time but I think we ended up generating some pdfs in the background and sending them by email when ready. The wicked_pdf gem uses an instance of https://github.com/wkhtmltopdf/wkhtmltopdf under the hood. This causes problems because it can only generate pdfs one by one. The solution would probably be having a dedicated service that would orchestrate multiple wkhtmltopdf instances.

We’ll start with the WickedPDF gem, which is powered by the wkhtmltopdf command-line library.

The only “sane” way I’ve found to be able to deal with pdfs is through this tool https://wkhtmltopdf.org/

wget + https://wkhtmltopdf.org/

Thanks for the suggestion! KnpSnappyBundle was my initial way to go as well, but my pages use quite some Javascript (chartJs) to render and I couldn’t get wkhtmltopdf to work with it. As it seems wkhtmltopdf does not support ES6 https://github.com/wkhtmltopdf/wkhtmltopdf/issues/3596 so I was forced to find another way.

wkhtmltopdf — Generates PDFs from HTML documents.

Of the HTML-based pdf-engines pandoc supports, prince would have the best typography, but I don't like recommending Prince because it's proprietary and costs money. (I try to stick to open source when I can.) wkhtmltopdf is the fastest, but uses a pretty old codebase, and doesn't even support paged/print css. weasyprint is a little better in my experience, but still has a ways to go typographically. pagedjs-cli is just a wrapper around headless Chrome/Chromium, and while Chrome has made improvements with regard to typography, Google turns off some of those features (e.g., hyphens) in headless mode, which is annoying.

Concerning pdf there is the well known wkhtmltopdf , but let me say that I love the not so well known percollate

Did you try wkhtmltopdf and WeasyPrint, by any chance?

Puppeteer is a Node.js library for automating web browsers. It is beneficial for scraping JavaScript-heavy websites.

[1]AITDK SEO Extension: https://aitdk.com/extension?utm_source=install [2]Driver.js: https://driverjs.com/ [3]shadcn/ui: https://ui.shadcn.com/ [4]Aceternity UI: https://ui.aceternity.com/ [5]Magic UI: https://magicui.design/ [6]Uiverse: https://uiverse.io/ [7]Apidog: https://apidog.com/ [8]Framer Motion: https://motion.dev/ [9]Vercel: https://vercel.com/home [10]Puppeteer: https://pptr.dev/

Headless Browsers: Tools like Puppeteer or Playwright help deal with JavaScript-heavy websites. These headless browsers can execute JavaScript and allow scrapers to interact with the page as if they’re a real user, helping to bypass complex obfuscation techniques.

Puppeteer: You’ll use this to control a headless browser.

The example setup from the blog post was a React app tested with Puppeteer and Jest. That’s a great start … but what if I use Playwright instead of Puppeteer? Is it possible?

If you are an expert in Node.js web automation, you might assume that using headless browsers controlled by technologies like Playwright or Puppeteer is more effective than utilizing curl-impersonate. No surprise, those two libraries are listed in our list of the best Node.js web scraping technologies. After all, browser automation tools also enable you to interact with the elements on the page. However, curl-impersonate is just an HTTP client that can only retrieve web pages. Still, there are Node.js web automation scenarios where a library like node-curl-impersonate might be a better choice than Playwright or Puppeteer. The reason for this is that anti-bot systems often use a two-step approach to detect and block bots. The first step checks if the request is coming from a legitimate browser, as explained earlier in this article. If the request seems suspicious, it is blocked. Otherwise, the server delivers the HTML document of the page. The page includes special JavaScript scripts that inspect the browser's settings and configurations to generate a browser fingerprint. This is then sent back to the anti-bot system to determine whether the user is legitimate. The second step works because automation tools tend to configure browsers in ways that differ from regular browsers. These differences are enough for anti-bot solutions to understand that they are dealing with an automated request. For more information, check out our guide on Playwright Extra. In contrast, curl-impersonate cannot render JavaScript, skipping the second step entirely. If the second step is not required to be considered a legitimate user, node-curl-impersonate can continue to effectively send requests to the target server without resource overheads and slowness typical of headless browsers — even in headles mode.

In this tutorial, you'll create a Trigger.dev task that scrapes the top 3 articles from Hacker News using BrowserBase and Puppeteer, summarizes them with ChatGPT and sends a nicely formatted email summary to yourself every weekday at 9AM using Resend.

We'll create another function which loads the page in a real browser and then extracts the rendered HTML content as soon as the page finished loading. We can use puppeteer to accomplish this:

In this blog, we’ll show you how to create a reliable Zalando scraper with Puppeteer - a well-known web scraping tool. You’ll learn how to pull out product details such as prices, sizes, and stock levels. We’ll also give you tips on how to handle CAPTCHA, IP blocking, and how to scale your scraper with Crawlbase Smart Proxy.

You could probably do it using something like Puppeteer [0].

[0]: https://pptr.dev/