multiserver VS websocketd

https://github.com/HimbeerserverDE/multiserver Just remembered that this exists. This may help you greatly since someone has already done the protocol work.

There's been several attempts to implement software like this in Minetest, the most prominent I know of is https://github.com/HimbeerserverDE/multiserver

Do you know how Minecraft servers solved the problem that MC is mostly single-threaded? They made server networks instead of single servers. There is a lobby from which players can pick on which sub-server they want to play on, and then are automatically redirected without having to reconnect. Will this be possible in Minetest at some point? Yes, and the point is now. There is a fully functional proxy named multiserver that allows for exactly that. The proxy still has quite a few bug and features yet to be implemented, but there are new commits almost every day. I am currently working on the Lua-side software for a server using this technology, it will go online soon (I'm not gonna say the name of this server since I don't want to advertise it, but rather just inform everyone of the latest developments.)

Somewhat related: https://github.com/joewalnes/websocketd

> websocketd is a small command-line tool that will wrap an existing command-line interface program, and allow it to be accessed via a WebSocket.

I hadn't even considered collecting traces/spans in this way yet, and have taken the approach of "stuff outputting logs in JSON format to stderr/local file". I usually end up writing a (temporary, structured) log message with the relevant span tags, but wouldn't it would be much better to run the actual trace/span code and be able to verify it locally without the ad-hoc log message?

The prototype I built is a web application that creates websocket connections, and if those connections receive messages that are JSON, log lines are added. Columns are built dynamically as log messages arrive, and then you can pick which columns to render in the table. If you're curious here's the code, including a screenshot: https://github.com/corytheboyd-smartsheet/json-log-explorer

With websockets, it's very easy to use websocketd (http://websocketd.com), which will watch input files for new lines, and write them verbatim as websocket messages to listeners (the web app).

To make the idea real, would want to figure out how to not require the user to run websocketd out of band, but watching good ol' files is dead simple, and very easy to add to most code (add a new log sink, use existing log file, etc.)

* Additionally, client can stream data to the backend server (if bidirectional GRPC streams are used). I.e. client sends WebSocket messages, those will be transformed to GRPC messages by WebSocket server and delivered to the application backend.

As a result we have a system which allows to quickly create individual streams by using strict GRPC contract but terminating connections over WebSocket transport. So it works well in web browsers. After that no need to write WebSocket protocol, client implementation, handle WebSocket connection. This all will be solved by a suggested WebSocket server and its client SDKs.

The mechanics is similar to Websocketd (https://github.com/joewalnes/websocketd), but instead of creating OS processes we create GRPC streams. The difference from grpc-web (https://github.com/grpc/grpc-web) is that we provide streaming capabilities but not exposing GRPC contract to the client - just allowing to stream any data as payload (both binary and text) with some wrappers from our client SDKs side for managing subscriptions. I.e. it's not native GRPC streams on the client side - we expose just Connection/Subscription object to stream in both directions. GRPC streams used only for communication between WebSocket server and backend. To mention - grpc-web does not support all kinds of streaming now (https://github.com/grpc/grpc-web#streaming-support) while proposed solution can. This all should provide a cross-platform way to quickly write streaming apps due to client SDKs and language-agnostic nature of GRPC.

I personally see both pros and cons in this scheme (without concentrating on both too much here to keep the question short). I spent some time thinking about this myself, already have some working prototypes – but turned out need more opinions before moving forward with the idea and releasing this, kinda lost in doubts.

My main question - whether this seems interesting for someone here? Do you find this useful and see practical value?

I also used to use this (http://websocketd.com/) along with netcat(1) before just biting the bullet and writing my own websocket library for our server as we needed to scale up slightly.

Another option if you’re ready to implement the frontend part is https://github.com/joewalnes/websocketd which has the advantage of streaming the output of your script

Wanted to add websocketd [1]. It's an amazing tool to stream debugging logs to another system where you can build your webapps that accumulate alerts.

Use it only for debugging builds and not for production (obviously).

[1] https://github.com/joewalnes/websocketd