CAP VS AngleSharp

You might want to implement the outbox pattern. There’s a decent library that’ll help and is really easy to integrate. In particular, I’d look at using the transaction functionality

This might be of interest https://cap.dotnetcore.xyz/

There are many libraries in .NET that helps you implementing the outbox pattern like: MassTransit, NServiceBus, CAP. Talking about idempotency, a special mention to a specific lib from a big friend that runs on top of CAP which is called Ziggurat.

CAP is your friend https://github.com/dotnetcore/CAP

Correct but I would actually consider CAP instead. The primary reason is that CAP implements the outbox pattern out-of-box where you would need to implement this yourself with Hangfire.

Instead, what I ended up doing is utilizing AngleSharp for transforming the existing (dynamic) websites into static files. I've stored them on disk and made them ready to be served statically.

AngleSharp

Read and understand the HTML DOM with something like this https://anglesharp.github.io/

Personally, if that script I wrote did need to be used on Core I probably would have compiled AngleSharp, loaded the HTML Parser DLL into the script, and used that.

Now, assuming you have to scrape, C# does have two decent scraping libraries: AngleSharp and Html Agility Pack. I personally prefer AngleSharp.

I would love a blog post or even a series of them about scraping local HTML files with either AngleSharp or HTMLAgilityPack.

If you want a cleaner and modern alternative to HtmlAgilityPack then I'd suggest https://github.com/AngleSharp/AngleSharp

Use anglesharp for parsing html