Show HN: Carton – Run any ML model from any programming language

1. carton

   1 5 422 8.4 Rust

   Run any ML model from any programming language.

   

   * Architecture - https://github.com/VivekPanyam/carton/blob/main/ARCHITECTURE...

   Please let me know what you think!

2. SaaSHub

   www.saashub.com featured

   SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

   

3. ivy

   2 17 14,178 8.0 Python

   Convert Machine Learning Code Between Frameworks

   

   is this ancillary to what [these guys](https://github.com/unifyai/ivy) are trying to do?

4. tfgo

   3 6 2,492 1.5 Go

   Tensorflow + Go, the gopher way

   

   eh, awesome! Seems this one, right? https://github.com/galeone/tfgo. Quite many stars.

5. arboreal

   4 1 8 2.9 Go

   pure Go library for gradient boosted decision trees

   

   We used Triton Inference Server (with a Golang sidecar to translate requests) for model serving and a separate Go app that handled receiving the request, fetching features, sending to Triton, doing other stuff with the response, serving. This scaled to 100k QPS with pretty good performance but does require some hops.

   In general writing pure Go inference libraries sucks. Not easy to do array/vector manipulation, not easy to do SIMD/CUDA acceleration, cgo is not go, etc. I wrote a fast XGBoost library at least (https://github.com/stillmatic/arboreal) - it's on par with C implementations, but doing anything more complex is going to be tricky.

Suggest a related project