websocketd VS yunohost

Somewhat related: https://github.com/joewalnes/websocketd

> websocketd is a small command-line tool that will wrap an existing command-line interface program, and allow it to be accessed via a WebSocket.

I hadn't even considered collecting traces/spans in this way yet, and have taken the approach of "stuff outputting logs in JSON format to stderr/local file". I usually end up writing a (temporary, structured) log message with the relevant span tags, but wouldn't it would be much better to run the actual trace/span code and be able to verify it locally without the ad-hoc log message?

The prototype I built is a web application that creates websocket connections, and if those connections receive messages that are JSON, log lines are added. Columns are built dynamically as log messages arrive, and then you can pick which columns to render in the table. If you're curious here's the code, including a screenshot: https://github.com/corytheboyd-smartsheet/json-log-explorer

With websockets, it's very easy to use websocketd (http://websocketd.com), which will watch input files for new lines, and write them verbatim as websocket messages to listeners (the web app).

To make the idea real, would want to figure out how to not require the user to run websocketd out of band, but watching good ol' files is dead simple, and very easy to add to most code (add a new log sink, use existing log file, etc.)

* Additionally, client can stream data to the backend server (if bidirectional GRPC streams are used). I.e. client sends WebSocket messages, those will be transformed to GRPC messages by WebSocket server and delivered to the application backend.

As a result we have a system which allows to quickly create individual streams by using strict GRPC contract but terminating connections over WebSocket transport. So it works well in web browsers. After that no need to write WebSocket protocol, client implementation, handle WebSocket connection. This all will be solved by a suggested WebSocket server and its client SDKs.

The mechanics is similar to Websocketd (https://github.com/joewalnes/websocketd), but instead of creating OS processes we create GRPC streams. The difference from grpc-web (https://github.com/grpc/grpc-web) is that we provide streaming capabilities but not exposing GRPC contract to the client - just allowing to stream any data as payload (both binary and text) with some wrappers from our client SDKs side for managing subscriptions. I.e. it's not native GRPC streams on the client side - we expose just Connection/Subscription object to stream in both directions. GRPC streams used only for communication between WebSocket server and backend. To mention - grpc-web does not support all kinds of streaming now (https://github.com/grpc/grpc-web#streaming-support) while proposed solution can. This all should provide a cross-platform way to quickly write streaming apps due to client SDKs and language-agnostic nature of GRPC.

I personally see both pros and cons in this scheme (without concentrating on both too much here to keep the question short). I spent some time thinking about this myself, already have some working prototypes – but turned out need more opinions before moving forward with the idea and releasing this, kinda lost in doubts.

My main question - whether this seems interesting for someone here? Do you find this useful and see practical value?

I also used to use this (http://websocketd.com/) along with netcat(1) before just biting the bullet and writing my own websocket library for our server as we needed to scale up slightly.

Another option if you’re ready to implement the frontend part is https://github.com/joewalnes/websocketd which has the advantage of streaming the output of your script

Wanted to add websocketd [1]. It's an amazing tool to stream debugging logs to another system where you can build your webapps that accumulate alerts.

Use it only for debugging builds and not for production (obviously).

[1] https://github.com/joewalnes/websocketd

Pull that old laptop from the closet, the one with the broken screen and keyboard which made you so sad to put it to pasture since it did have plenty of memory and CPU to keep up. Install Debian on the thing followed by Proxmox Virtual Environment (PVE) [1]. Since you have 16GB of RAM in that laptop (or 8 but 16 is nicer) you should be able to run a number of containers [2].

Here's an idea, more or less based on a number of servers I configured for friends and family, based on 8GB Raspberry Pi 4 hardware with 2/4TB USB SSD. Your laptop will offer better performance.

- Create 4 or 5 containers and name them 'auth', 'serve´, 'base', 'backup' and 'mail' (if you want to run your own mail that is, otherwise skip that one). Their functions are:

> auth runs LDAP, Kerberos (if you want that), a central letsencrypt instance which takes care of all your certificate needs and anything else related to authentication and authorisation

> base runs databases, that means Postgresql, Mysql/Mariadb, Redis, RabbitMQ and whatnot - all depending on what you need.

> serve runs services, that means nginx or another web server which is used as a reverse proxy for the other web-related things you want to run: 'cloud' services like Nextcloud with everything that comes with it (e.g. Collaboraoffice or Onlyoffice to replace whatever web-based office things you currently use), communications services like XMPP, application-specific proxies like Invidious/Nitter/Libreddit, media services like Peertube/Airsonic/Ampache, a Wiki like Bookstack, search services like SearxNG, etc. - the size of your server is the limit.

> backup runs Proxmox Backup Server and is used to backup everything to some external drive and to some outside repository.

> mail runs mail services, only if you want to run those. I always say 'do it' but many people have an irrational fear of running their own mail services. That fear is not grounded in truth, running mail is not hard and offers many advantages over hosted solutions.

While it is possible to separate all the mentioned services out into their own containers I think this adds needless complexity for little to no gain. Separating out database services makes sense since those can end up quite taxing and as such might well be moved to their own hardware in some (possibly not too distant) future. Separating out authentication services makes sense since that lowers the attack surface compared to running them together with externally available services. The same goes for mail services which is why I put those in their own container.

Once you've got this up and running you can create a few more containers to play around with. If you just want to try out services something like Yunohost [3] or Caprover [4] can come in handy but I do not see these as viable alternatives to installing and running services which you intend to keep around for a long time.

Of course you can do most of this on a VPS as well but I prefer to keep thing in-house - the fewer dependencies, the better.

[1] https://proxmox.com/en/

[2] containers perform better and take less memory than VMs but if VMs are your thing that is possible as well

[3] https://yunohost.org

[4] https://caprover.com/

Yunohost is one of those mature projects, that's fully open source.

YunoHost, although not Docker-based, is still nice and quite mature.

If you want something like that, then CasaOS is pretty great and i can recommend it, especially for a beginner. There is also Cosmos and Tipi. Yunuhost too but a bit different approach. Oh and Umbrel is a thing...

However you quickly reach the limits of what Umbrel can do, its very basic in its abilities. Of course it depends all on what you (or anyone else) wants to do with it. There is also CasaOS which is very similar to Umbrel but last i compared, Casa offered a bit more features like for example adding your own docker projects easily. There is also Tipi which i must admit i havent taken a closer look at yet. And there is Yunohost which i guess aims at a similar audience but achieves these things differently, still worth mentioning tho.

This looks exciting and definitely something to look out for as an option fkr self-hosting.

Similiar and a little bit more mature is also YunoHost, https://yunohost.org/, or for professional environments, UCS https://www.univention.com/.

I use https://yunohost.org on my Pi, mostly for monitoring other stuff but you can get Nextcloud running just fine with it!