mediapipe VS tonic

mediapipe/docs/solutions/pose.md at master · google/mediapipe · GitHub

Today, we announce MediaPipe diffusion plugins, which enable controllable text-to-image generation to be run on-device. Expanding upon our prior work on GPU inference for on-device large generative models, we introduce new low-cost solutions for controllable text-to-image generation that can be plugged into existing diffusion models and their Low-Rank Adaptation (LoRA) variants.

You can just grab the TFLite version! https://github.com/google/mediapipe/blob/master/docs/solutions/models.md

I believe Google already released transformers under an apache 2 license with a patent grant:

https://github.com/google/mediapipe/blob/master/mediapipe/mo...

I was going to say that I like the MediaPipe Selfie Segmentation model for doing this sort of thing in a web page, but I've just noticed (when getting the GitHub link[1]) that Google have marked the code as legacy[2] ... no idea if the new solution is better/easier to use[3].

For what it's worth, my CodePen using the old model is here: https://codepen.io/kaliedarik/pen/PopBxBM

[1] - https://github.com/google/mediapipe/blob/master/docs/solutio...

[2] - "Attention: Thank you for your interest in MediaPipe Solutions. As of April 4, 2023, this solution was upgraded to a new MediaPipe Solution."

[3] - https://developers.google.com/mediapipe/solutions/vision/ima...

I have used mediapipe very successfully in multiple projects and it's very easy to get running. You can choose from many different vision tasks including hand landmarks ( https://github.com/google/mediapipe/blob/master/docs/solutions/hands.md )

I found MediaPipe's Face Mesh and was impressed with how simple it was to get going, but it just gives you the landmark points and I've not gone any further yet.

We've trained ControlNet on a subset of the LAION-Face dataset using modified output from MediaPipe's face mesh annotator to provide a new level of control when generating images of faces.

Medipaipe library link: https://mediapipe.dev/

Use tonic-build directly from Rust.

} // Omit the remaining code and refer to the example in Tonic: https://github.com/hyperium/tonic/blob/master/examples/src/multiplex/server.rs ```

If I’m not misunderstanding, you could both decode the gRPC protobuf AND write to delta lake in Rust. Tonic, Delta-rs.

Original post: https://github.com/hyperium/tonic/issues/1405. Cross-post here in case the problem is not specific to tonic.

Tonic

Just to double check Is this the correct repo?

Wasm isn't involved much actually. Basically, the frontend asks the backend to perform a gRPC call on its behalf using Tauri events. They are like named channels on which you can send any serde-compatible value. But the backend is a normal Rust program so there are no constraints there. I use prost-reflect to encode/decode Protobuf messages according to Protobuf descriptors loaded at runtime and make the actual requests using tonic from the tokio ecosystem. prost-reflect is necessary because, normally, tonic expects the Protobuf descriptor to be known at compile time so it can make some code generation behind the scenes.

You could have a look at gRPC i.e. https://github.com/hyperium/tonic

Looking at this example code: https://github.com/hyperium/tonic/blob/master/examples/src/tower/server.rs

The remaining gap is remote actors, since you still need some kind of serialization between them, and take your pick of standards for that one such as gRPC using Tonic.