os-nvr VS coral-pi-rest-server

I'm working on this. RTSP and HLS restreaming support, super reliable non-existing database.

GitLab Screenshots Change log Matrix chat

GitLab ⭐ Sceenshots

OS-NVR

OS-NVR - in Go (https://gitlab.com/osnvr/os-nvr)

OS-NVR

However, a pi doesn't have the strength to run something like Llama.cpp, of course, so I've been considering using something like the Coral USB Accelerator (https://coral.ai/products/accelerator). As I've been learning more about it, it seems to be very geared towards TensorFlow Lite models. But whisper.cpp and Llama.cpp use ggml models.

> On a Raspberry Pi 4, Piper can generate 2 seconds of audio with only 1 second of processing time.

> On a Raspberry Pi 4, voice commands can take around 7 seconds to process with about 200 MB of RAM used.

Have you looked into supporting something like the Coral Accelerator[0], which can drastically speed up machine learning inference on a Raspberry Pi?

[0]: https://coral.ai/products/accelerator

Yeah it's a USB stick. https://coral.ai/products/accelerator/ Price came down to $60

These ones can be plugged in with USB type c.

https://coral.ai/products/accelerator/

Used for boosting interference (offline) on Linux, Mac and Windows

Haven’t bought or used them but I’ve had my eyes on these for a little while!

What do you think would be the best hardware and software suited for this if we should go this route? I've been reading up on Frigate and Google Coral, but I am unclear on how it all fits together, the hardware required, etc, if that makes sense. And then what about an Android app to send notifications and view footage? Or would I set up postfix to send emails as notification? I am unsure of how this all would work.

In this case we are talking about https://coral.ai/products/accelerator/ basically it's a "coprocessor" for AI neural network. You can train your models on a beefy computer with proper GPU for speed... but the trained AI network can then be used on some edge low power devices. But to process already trained network in any resemblance of real time, you can't use CPU ( too slow even on big PCs), GPU (Graphic card can't fit to Raspberry Pi, or smaller edge devices ) therefore TPU, a USB dongle like device, that took the AI processing part out of graphic card (on smaller scale) and allows you to execute AI stuff directly on that HW with almost real time performance.

If I didn't have a Home Assistant installation I'd bring up a small virtual machine on my Blue Iris server and pass the USB/PCIE coral device through to the VM and install this coral-pi-rest-server (same underlying code I'm running on my Home Assistant add-on). I bought some PCIE TPUs as well but they haven't shipped yet so I won't be testing this method until next year it seems.