srs VS hls4ml

With OBS supporting HEVC/H.265 (and AV1, VP9) as output codec in a recent beta, I've been testing VRChat support. Keeping eye on https://github.com/ossrs/srs/pull/3495 before I try HEVC livestreams into VRChat as I believe I can try RTSP and mpeg-ts.

For in-house hosting, a project like https://github.com/ossrs/srs or https://github.com/illuspas/Node-Media-Server may meet your needs. Ultimately though, you should be able to build whatever you need from scratch using an Nginx server with the RTMP module and an ffmpeg process

I don't know the answer to the WebRTC part, but as long as you have a server with not-outrageously-priced outbound bandwidth, you can install an open source RTMP server like SRS[1], and stream to that RTMP server from OBS. It's really easy, configure the RTMP server & stream key, then "Start Streaming" which is right next to "Start Recording". You can then hand your friends a link, and they can play it in any media player with RTMP/HLS/FLV stream support, or you can add a simple web UI with e.g. hls.js[2] (very easy to write, there might even be prepackaged solutions) so that they truly don't need to download anything.

[1] https://github.com/ossrs/srs

[2] https://github.com/video-dev/hls.js/

not flask related but SRS is my go to when i think about media/video streaming

from the configure file, it looks like there should be a make destroy command which basically just removes everything in /objs: https://github.com/ossrs/srs/blob/4.0release/trunk/configure

SRS

I see that someone has already suggested hls4ml. I second that opinion. From my experience, it is extremely well documented. They have published papers which explain the scientific background. They have a really nice git page where they explain all the features of their tool. Additionally they also have an easy to follow tutorial of doing it from scratch using tensorflow networks. You can find all the information herehls4ml.

Open source project that originated at Fermilab https://github.com/fastmachinelearning/hls4ml (based on Xilinx Vivado which has been replaced by Vitis)

You may check the hls4ml framework: it's a "translator" from the ML model (Keras, PyTorch) to a synthesizable High-Level Synthesis (HLS) IP Core.

It really depends on the type of data you are using. There may (or may not) be some trade offs and sacrifices. There are frameworks which can basically translate your neural network information from a high level python code into equivalent HLS code which is optimized for low latency when inferred on FPGAs. Some frameworks which might be useful for you to explore are hls4ml and finn. These are some frameworks which can achieve low latency inference of neural networks on FPGAs using Xilinx Vitis HLS. These are what I found when I did a similar experiment but with much lower latency target (a few hundred ns) and a very simple MLP with 1D signal as input which was a year ago. Not sure if there are better alternatives available as of 2023. But conceptually all these work on the primary principle of having a supporting framework/methodology to first quantize the network and limit the precision of data to fixed point. The HLS then produced will also be a result of the framework applying dataflow techniques such that the resulting HLS code will produce an RTL which has the best overall latency.

Check hls4ml. Developed by CERN for fast classification in FPGA for high-energy physics experiments.

I would check out hls4ml. It's an open source project made by/for people at CERN to convert neural networks created in Python using QKeras (a quantization extension of Keras) into HLS, with Vivado HLS being the most well supported. There are some caveats though, and a fellow student and I have had trouble getting the generated HLS to match the Keras model and be feasible to synthesize, but it seems to work well for smaller neural networks.

Since you're looking for PL-only implementation, HLS4ML may fit your needs. It was developed to port TensorFlow models directly to FPGAs in particle physics experiments. Current development allows for implementation on SoC and MPSoC, though.