selenium-powershell VS WKHTMLToPDF

There is a powershell module for it as well Selenium-powershell

If the website is using a JS library like React and the spinning you’re referring to is the JS library trying to pull data back to drop into the DOM then you can look at the Selenium PowerShell module to see if expected HTML gets dropped in. Info on the PowerShell module: https://github.com/adamdriscoll/selenium-powershell

I'm trying to access a website that is protected by Cloudflare. Whenever I use Invoke-WebRequest, I get a 403 error. Even if I use Selenium, I get a checkbox to verify that I am human. Is there any way to access the website using Powershell or is Cloudflare designed to stop this type and similiar requests?

I've used Selenium and PowerShell to do it in the past.

I've had great success using selenium in powershell 5.1 (it was very slow with ps core for some reason): https://github.com/adamdriscoll/selenium-powershell

for PS + Selenium, review the following: https://github.com/adamdriscoll/selenium-powershell/blob/master/Examples/A-Simple-Google-Search.ps1

If you want to do this in PowerShell, I'd start by looking at the popular selenium module: Here is the first search result I found for this topic: https://saucelabs.com/resources/articles/print-page-pdf. Here is the PowerShell repo: https://github.com/adamdriscoll/selenium-powershell

This module and the included browser drivers are great for automating browser stuff https://github.com/adamdriscoll/selenium-powershell

Powershell module from adamdriscoll/selenium-powershell: PowerShell module to run a Selenium WebDriver. (github.com) but the problem also persists if i load the driver manually in powershell.

We’ll start with the WickedPDF gem, which is powered by the wkhtmltopdf command-line library.

The only “sane” way I’ve found to be able to deal with pdfs is through this tool https://wkhtmltopdf.org/

wget + https://wkhtmltopdf.org/

Thanks for the suggestion! KnpSnappyBundle was my initial way to go as well, but my pages use quite some Javascript (chartJs) to render and I couldn’t get wkhtmltopdf to work with it. As it seems wkhtmltopdf does not support ES6 https://github.com/wkhtmltopdf/wkhtmltopdf/issues/3596 so I was forced to find another way.

wkhtmltopdf — Generates PDFs from HTML documents.

Of the HTML-based pdf-engines pandoc supports, prince would have the best typography, but I don't like recommending Prince because it's proprietary and costs money. (I try to stick to open source when I can.) wkhtmltopdf is the fastest, but uses a pretty old codebase, and doesn't even support paged/print css. weasyprint is a little better in my experience, but still has a ways to go typographically. pagedjs-cli is just a wrapper around headless Chrome/Chromium, and while Chrome has made improvements with regard to typography, Google turns off some of those features (e.g., hyphens) in headless mode, which is annoying.

Concerning pdf there is the well known wkhtmltopdf , but let me say that I love the not so well known percollate

Did you try wkhtmltopdf and WeasyPrint, by any chance?

Since the srd uses standardized links (base/creature type/creature name) you could make a list of urls based on your selected monsters in a spreadsheet, then use a program like https://wkhtmltopdf.org/, https://www.weenysoft.com/free-html-to-pdf-converter.html, or the url conversion feature in Adobe Acrobat Pro if you combine all the urls into an htm for Acrobat to pull from.