njs VS replace-response

This article came at an interesting timing for me, since I recently started to explore building my own CDN node on top of NGINX open source inspired by this article from Fly: https://fly.io/blog/the-5-hour-content-delivery-network/

I've worked with nginx in the past, and didn't have a great experience, so I was apprehensive diving in, but this time was very different. I think njs (their custom JS scripting environment) was a game changer. Support is built in to nginx core, and available by default in their docker containers, so it's much easier to get started with than Lua scripting. Their JS feature support has some quirks (no optional chaining, array destructuring, console.log's don't show up in logs, are some examples of things that threw me off) but overall nothing that blocked me from implementing the functionality I needed, and the integration points within the nginx config felt fairly natural.

I did run into a number of things that were locked behind their commercial offering that made me a bit uncomfortable betting on it for the long term compared to purely open source alternatives. Off the top of my head:

- DNS discovery. There's a thread on the Fly example repo accompanying the blog post that describes the use case and proposes some workarounds: https://github.com/fly-apps/nginx-cluster/issues/2. Life would be a lot simpler if DNS discovery from the commercial offering was just available (i.e. we can outright delete a brittle bash script that makes DNS queries and reloads nginx on a 5 second interval). This was mentioned in the article as something they're planning to open source.

- Access to some kind of shared key-value store for custom caching logic in njs scripts. With Lua we could just connect to Redis, but njs can't seem to establish persistent network connections for now, so that's off the table. This wasn't mentioned in the article, but they did mention in this Github issue that they're planning on open sourcing their keyval module for this use case: https://github.com/nginx/njs/issues/437. I have some use cases where being able to connect to Redis would be ideal, since syncing keyval across a cluster seems to be eventually consistent (https://docs.nginx.com/nginx/admin-guide/high-availability/z...), but for most of my caching use cases it should be sufficient.

So this article, along with their overall willingness to work with the community to identify and bring commercial features into open source (at least from what I've observed across their responses to Github issues) does a lot to alleviate those concerns.

Though at the end of the day, I don't necessarily need every nginx feature to be in open source. I have no problems with paying for great software like nginx to support its development. But as a small bootstrapped founder, their current pricing structure (from what I could gather on the internet is ~ 2k-5k per running instance), is completely prohibitive. It'd probably require a revamp to the way they sell the software (i.e. self-serve onboarding and automatic license provisioning for smaller customers instead of having customers of all sizes go through expensive sales people), but I'd love to see a more progressive pricing structure with a lower barrier to entry for their commercial product.

I'd leave guys a bug at github: https://github.com/nginx/njs .

> But there are many scenarios where being able to extend the HTTP server via Lua is more convenient than writing a plugin I would think?

Well, Caddy is written in Go, so it's only natural to write a plugin in Go. Statically compiled into your binary. We provide a tool called `xcaddy` which is used to produce builds of Caddy with any plugins you need. You just need Go installed on your system to run it, no other dependencies.

The reason why Lua is used for OpenResty is because writing plugins in C is... not fun.

You can absolutely do what you described with an HTTP handler module in Caddy. You'd just wrap the req.Body with a reader that watches the bytes as they're copied through the stream, and when you see the part you want to log, you do that.

We have a replace-response plugin which takes a similar approach, except it manipulates the response as it's being streamed back to the client. https://github.com/caddyserver/replace-response The whole plugin is just one file of Go code.