AngleSharp VS playwright-sharp

Instead, what I ended up doing is utilizing AngleSharp for transforming the existing (dynamic) websites into static files. I've stored them on disk and made them ready to be served statically.

AngleSharp

Read and understand the HTML DOM with something like this https://anglesharp.github.io/

Personally, if that script I wrote did need to be used on Core I probably would have compiled AngleSharp, loaded the HTML Parser DLL into the script, and used that.

Now, assuming you have to scrape, C# does have two decent scraping libraries: AngleSharp and Html Agility Pack. I personally prefer AngleSharp.

I would love a blog post or even a series of them about scraping local HTML files with either AngleSharp or HTMLAgilityPack.

If you want a cleaner and modern alternative to HtmlAgilityPack then I'd suggest https://github.com/AngleSharp/AngleSharp

Use anglesharp for parsing html

Or https://github.com/microsoft/playwright-sharp

There are multiple ways to do this. For web scraping like you describe, I would use Playwright-sharp.

I've used PuppeteerSharp for web automation with great success. I understand that project is now Playwright-Sharp. I would highly recommend that over something like Selenium which is really more for testing.

Using either Puppeteer or Playwright to scrape a web page is super simple. Both have their own .NET ported versions like Puppeteer Sharp and Playwright Sharp respectively. However, they don't work on Azure Functions, unfortunately. More precisely, they work on your local dev environment, not on Azure instance. This post would be useful for your node.js Azure Functions app, but it's not that helpful for your .NET application. Let me find a way for it to work on Azure Functions instance correctly.