blindai VS tract

We are working on an open-source project, BlindAI (https://github.com/mithril-security/blindai) to answer exactly that: privacy when sending data to remote AI models.

BlindAI (https://github.com/mithril-security/blindai) is an open-source confidential AI deployment. By using secure enclaves (Intel SGX for now, soon AMD SEV and Nvidia Confidential Computing), we provide end-to-end protection for users’ data, even when sending it to the Cloud for AI inference. You can see the gains of BlindAI on the scheme below:

Good thing with enclave is that the hardware protection enable us to use regular AES to secure communication with the enclave, which means no ciphertext expansion and lightweight client side. We do not need to have a complicated client side, we just need a slightly modified TLS client with additional security checks, like remote attestation but you can have a look on our client side it's light (https://github.com/mithril-security/blindai/tree/master/client).

I am glad to introduce BlindAI, an AI deployment solution, leveraging secure enclaves, to make remotely hosted AI models privacy friendly. We leverage the tract project as our inference engine to serve AI models in ONNX format inside an enclave. We also use the Rust SGX SDK to use Rust for our secure enclave for AI.

Tract is the most well known ML crate in Rust, which I believe can compile to WASM - https://github.com/sonos/tract/. Burn may also be useful - https://github.com/burn-rs/burn.

tract!!

we use tract for inference, integrated into our runtime and services.

Not for training, but for inference this looks nice: https://github.com/sonos/tract

There's also tract, from sonos[0]. 100% rust.

I'm currently trying to use it to do speech recognition with a variant of the Conformer architecture (exported to ONNX).

The final goal is to do it in WASM client-side.

[0] https://github.com/sonos/tract

As the article notes, there isn't any official Rust-native support for any common frameworks.

tract (https://github.com/sonos/tract) seems like the most mature for ONNX (for which TF/PT export is good nowadays), and recently it successfully implemented BERT.

There are some DL libraries written in Rust: https://github.com/sonos/tract , https://docs.rs/neuronika/latest/neuronika/index.html . The second one could be used for training, I think.