onnxruntime-rs VS blindai

I did: https://github.com/nbigaouette/onnxruntime-rs/pull/87 but the maintainer seems to be off. I sent an email.

I have used the https://github.com/nbigaouette/onnxruntime-rs ONNX C++ wrapper on a Pytorch model, and did not see any difference in compute time between ONNX Python and ONNX Rust for GPU.

We are working on an open-source project, BlindAI (https://github.com/mithril-security/blindai) to answer exactly that: privacy when sending data to remote AI models.

BlindAI (https://github.com/mithril-security/blindai) is an open-source confidential AI deployment. By using secure enclaves (Intel SGX for now, soon AMD SEV and Nvidia Confidential Computing), we provide end-to-end protection for users’ data, even when sending it to the Cloud for AI inference. You can see the gains of BlindAI on the scheme below:

Good thing with enclave is that the hardware protection enable us to use regular AES to secure communication with the enclave, which means no ciphertext expansion and lightweight client side. We do not need to have a complicated client side, we just need a slightly modified TLS client with additional security checks, like remote attestation but you can have a look on our client side it's light (https://github.com/mithril-security/blindai/tree/master/client).

I am glad to introduce BlindAI, an AI deployment solution, leveraging secure enclaves, to make remotely hosted AI models privacy friendly. We leverage the tract project as our inference engine to serve AI models in ONNX format inside an enclave. We also use the Rust SGX SDK to use Rust for our secure enclave for AI.