xvc VS upspin

SEEKING WORK | REMOTE | Istanbul | UTC Business Hours

ML / MLOps / Data Engineering

Looking for projects that I can contribute in machine learning, data management, data pipelines, and similar technologies.

These days, I'm building an MLOps tool to manage data and pipelines on top of Git, in Rust.

https://github.com/iesahin/xvc (https://docs.xvc.dev)

I'm experienced in libraries like Tensorflow and PyTorch, languages (Rust, Python, Go, Dart, C, ...) and Docker, AWS, GCP. I used Vue.js & Flask for web, and Flutter for mobile in the past but these are not my current focus now.

I'm also a technical writer, and studying copywriting and marketing these days.

I can also architect your software.

https://emresahin.net/cv/ (I'm in the middle of updating the site but gives you an idea.)

[email protected]

Working on a new MLOps tool to manage files, data, pipelines, experiments and models.

I'm writing it in Rust. Using ECS instead of OOP. Going well so far.

It's open (and alpha.) https://github.com/iesahin/xvc

Documentation: https://docs.xvc.dev

Super intriguing. Thanks for sharing!

It reminds me a bit of an early Go project called Upspin [1]. And also a bit of Solid [2]. Did you get any inspiration from them?

What excites me about your project is that you're addressing the elephant in the room when it comes to data sovereignty (~nobody wants to self-host a personal database but their personal devices aren't publicly accessible) in an elegant way.

By storing the data on my personal device and (presumably?) paying for a managed relay (and maybe an encrypted backup), I can keep my data in my physical possession, but I won't have to host anything on my own. Is that the idea?

https://upspin.io/

There are a few Go projects meant to be learned from:

- https://github.com/pion/opus for to learn audio

- https://github.com/benbjohnson/wtf for overall production quality

- https://github.com/upspin/upspin difficult to explain, personally I'm not a fan of the errors

You could also take a look at some real-world open-source projects. I like upspin for its idiomatic approach.

For example, Rob Pike's upspin places all its validations in the separate package. Do you agree with that approach? Which yet proven options there are?

Just a few projects that could perhaps interest you in terms of design of your own solution :

Upspin: https://upspin.io/

The early error wrapping work which emerged out of the Upspin project, that eventually made its way into the errors package, included stack traces in the wrap error. This would provide exactly what it appears you seek.