puppeteer-cluster VS deno-puppeteer

If urlsToScrape happens to be large enough, the memory and processor load from spawning dozens or hundreds of pages can bring a system down quickly. Consider puppeteer-cluster.

You could set up a service with something like playwright, puppeteer puppeteer-cluster, browserless, to access the service internally and serve screenshots of it to the outside user. You'd have to set up probably some kind of web service with the appropriate routes.

To make Unlighthouse fast, we combined this lighthouse binary with the package puppeteer-cluster, which allows for multi-threaded lighthouse scans.

https://github.com/thomasdondorf/puppeteer-cluster is doing queue-like mechanism but it is not using redis.

I've used this library: Puppeteer cluster

In an ideal world, yes, but in reality Deno's permission model is quite hard to use so many libraries require disabling a lot of it, sometimes everything like puppeteer. Other example is the library I maintain: it requires --allow-net since you can't whitelist a domain and all its subdomains, just a domain.

import puppeteer from "https://deno.land/x/[email protected]/mod.ts"; const browser = await puppeteer.launch(); const page = await browser.newPage(); await page.goto("https://news.ycombinator.com", { waitUntil: "networkidle2", }); await page.pdf({ path: "hn.pdf", format: "A4" }); await browser.close();

I agree that semantics for that is complicated, but I think the adopting browser permission model is bad because browsers have per-site isolation but how Deno is going to do that for applications that require executing external binaries? Also, even if you would whitelist some binaries, there are Deno packages, such as deno-puppeteer, which don't list what permissions it requires, instead it instructs users to enable all permissions using -A. By the way, why just -A? Why not --unsafely-enable-all-permissions (like Chromium's --unsafely-treat-insecure-origin-as-secure)?

Maybe people forget about this permission system because either are not experienced with Deno or because they just slap -A on eveything. Some packages such as deno-puppeteer even put it in all examples without even adding a note about its risks.