flink-kubernetes-operator VS Nginx

Data scientists often prefer Python for its simplicity and powerful libraries like Pandas or SciPy. However, many real-time data processing tools are Java-based. Take the example of Kafka, Flink, or Spark streaming. While these tools have their Python API/wrapper libraries, they introduce increased latency, and data scientists need to manage dependencies for both Python and JVM environments. For example, implementing a real-time anomaly detection model in Kafka Streams would require translating Python code into Java, slowing down pipeline performance, and requiring a complex initial setup.

Other stream processing engines (such as Flink and Spark Streaming) provide SQL interfaces too, but the key difference is a streaming database has its storage. Stream processing engines require a dedicated database to store input and output data. On the other hand, streaming databases utilize cloud-native storage to maintain materialized views and states, allowing data replication and independent storage scaling.

also, this knowledge applies to learning more about data engineering, as this field of software engineering relies heavily on the event-driven approach via tools like Spark, Flink, Kafka, etc.

Apache SeaTunnel is a data integration platform that offers the three pillars of data pipelines: sources, transforms, and sinks. It offers an abstract API over three possible engines: the Zeta engine from SeaTunnel or a wrapper around Apache Spark or Apache Flink. Be careful, as each engine comes with its own set of features.

Due to the technology transformation we want to do recently, we started to investigate Apache Iceberg. In addition, the data processing engine we use in house is Apache Flink, so it's only fair to look for an experimental environment that integrates Flink and Iceberg.

Yeah, unless I'm looking at it wrong, there doesn't seem to be any meaningful difference between 1.25.5 and 1.26.0:

https://github.com/nginx/nginx/compare/release-1.25.5...rele...

However, it's very unlikely that .NET developers will directly expose their Kestrel-based web apps to the internet. Typically, we use other popular web servers like Nginx, Traefik, and Caddy to act as a reverse-proxy in front of Kestrel for various reasons:

> I actually don't understand why I am seeing arguments like this all the time.

Have a look at:

https://github.com/nginx/nginx/blob/master/src/http/modules/...

It's got the whole checklist: nginx idiosyncratic module system, inline parsing, custom utf conversion, buffer preallocation and adjustments, linked lists, comments about side effects of custom allocator, and probably other things.

It's not easy to deal with source like that and any serious improvement to that area would effectively be a rewrite anyway.

Since anything doing work in nginx is a module anyway, it wouldn't even have to be a full rewrite in one go.

According to this article, nGinx is being used to serve 34% of all websites in the world. I checked out who's contributing to nGinx, and just like I thought, the project has 8,208 commits, and 5,366 of those commits was made by 2 software developers; igorsoev and mdounin.

Nginx + Roadrunner (fcgi mode)

Apache’s mod_fastcgi’s last commit was 2 weeks ago:

https://svn.apache.org/viewvc/httpd/httpd/trunk/

It’s a fork of what you linked (and was more popular afaik back when fastcgi was state of the art, and apache was the undisputed champion of web servers).

These days, nginx has more market share than apache, and its fastcgi module is one of the more recently updated ones in its source tree (5 months vs multiple years):

https://github.com/nginx/nginx/tree/master/src/http/modules

If I was going to build an embedded web server, I’d start with nostd rust, probably with though axum + tokio, since thats already memory safe-ish.

If I needed fastcgi for some reason (dynamically loadable endpoints, or os-level isolation), there are at least four implementations of fastcgi for it. No idea if any are decent though.

APISIX is an API Gateway. It builds upon OpenResty, a Lua layer built on top of the famous nginx reverse-proxy. APISIX adds abstractions to the mix, e.g., Route, Service, Upstream, and offers a plugin-based architecture.