souin VS haproxy

HTTP plugin now support RFC 7234 caching options

It's always been blisteringly fast when we've used it, and I like the power of the configuration (it has its quirks but so do most powerful systems). But the overhead of setting it up and maintaining it due to having to handle TLS termination separately puts me off using it when other software is 'good enough'. If Varnish Enterprise was cheaper I would have bought it, but at their enterprise prices no way.

I'm keeping a watching brief on https://github.com/darkweak/souin and its Caddy integration to see if that can step up and replace Varnish for short-lived dynamic caching of web applications. Though I've lost track of its current status.

Centminmod (need lots of tweaks) or OpenLiteSpeed are consider more performant than default Nginx and Caddy, but I wish if someone can prove there is a better one out there or I just saw Souin HTTP cache (https://github.com/darkweak/souin) when I was reading Echo web framework for Go. Yet to experiment if there is a benefits unless it's insignificant.

I wanted to try it out just now but hit a roadblock immediately - it cannot automatically obtain and maintain TLS certificates. You have to use an external client (e.g. acme.sh), set up a cron to check/renew them, and poke HAProxy to reload them if necessary. I'm way past doing this in 2023.

https://www.haproxy.com/blog/haproxy-and-let-s-encrypt

https://github.com/haproxy/haproxy/issues/1864

For context I think this might be useful: https://github.com/haproxy/haproxy/blob/master/addons/promex/README

I turn on 1 relatively cheap cloud server to process DNS, https and stratum connections and route them via haproxy to one of N miner servers described above.

Full technical writeup here: https://github.com/haproxy/haproxy/commit/a8598a2eb11b6c989e81f0dbf10be361782e8d32

With the recent discussions about memory safe languages, HAProxy is still surprisingly written in C [0].

[0]: https://github.com/haproxy/haproxy

It does not, because HAProxy does not perform any disk access at runtime and thus would be unable to persist the certificates anywhere. Disks accesses can be unpredictably slow and would block the entire thread which is not something you want when handling hundreds of thousands of requests per second.

See this issue and especially the comment from Lukas Tribus: https://github.com/haproxy/haproxy/issues/1864

Disclosure: Community contributor to HAProxy, I help maintain HAProxy's issue tracker.

Code link: https://github.com/haproxy/haproxy