coral-pi-rest-server VS HASS-coral-rest-api

That said, you can always look at something like https://coral.ai/products/accelerator/ to help with the performance you need.

Should only be ~$60 https://coral.ai/products/accelerator/

The Coral USB accelerator might be of interest if you want to do some light ML with a low power budget.

However, a pi doesn't have the strength to run something like Llama.cpp, of course, so I've been considering using something like the Coral USB Accelerator (https://coral.ai/products/accelerator). As I've been learning more about it, it seems to be very geared towards TensorFlow Lite models. But whisper.cpp and Llama.cpp use ggml models.

Recently found this thing, though I haven't found a use case for me.

I would start here: https://github.com/grinco/HASS-coral-rest-api

This was about a 10X improvement for me using Google Coral TPU REST API for HASS (coral-pi-rest-server) from Home Assistant (different physical hardware). It seems like native Blue Iris Coral support could shave off even a few more milliseconds.

When I eventually get one my plan is to either set up a standalone machine and utilize a http rest server shim that uses the same interface as deepstack https://github.com/robmarkcole/coral-pi-rest-server or if I can get it to work in my existing home assistant server there's this addon: https://github.com/grinco/HASS-coral-rest-api. Then you essentially just point Blue Iris at this "Deepstack/SenseAI" server.