xvc VS FluidFramework

SEEKING WORK | REMOTE | Istanbul | UTC Business Hours

ML / MLOps / Data Engineering

Looking for projects that I can contribute in machine learning, data management, data pipelines, and similar technologies.

These days, I'm building an MLOps tool to manage data and pipelines on top of Git, in Rust.

https://github.com/iesahin/xvc (https://docs.xvc.dev)

I'm experienced in libraries like Tensorflow and PyTorch, languages (Rust, Python, Go, Dart, C, ...) and Docker, AWS, GCP. I used Vue.js & Flask for web, and Flutter for mobile in the past but these are not my current focus now.

I'm also a technical writer, and studying copywriting and marketing these days.

I can also architect your software.

https://emresahin.net/cv/ (I'm in the middle of updating the site but gives you an idea.)

[email protected]

Working on a new MLOps tool to manage files, data, pipelines, experiments and models.

I'm writing it in Rust. Using ECS instead of OOP. Going well so far.

It's open (and alpha.) https://github.com/iesahin/xvc

Documentation: https://docs.xvc.dev

Have you seen FluidFramework? It's open source (MIT): https://github.com/microsoft/FluidFramework

I think the first product they're building on it is Loop: https://www.zdnet.com/article/microsoft-introduces-loop-a-ne...

Full disclosure I work at MSFT and on the fluid framework.

If you are interested in this you may also be interested in the fluid framework, https://github.com/microsoft/FluidFramework

We use websockets and solve a lot of the state management problem called out here by keeping very little state on the server itself. The primary thing on server is a monotonically increasing integer we use to stamp messages, this gives us total order broadcast which we then build upon: https://en.m.wikipedia.org/wiki/Atomic_broadcast

Here are some code pointers if you want to take a look:

The map package is a decent place to look for how we leverage total order broadcast to keep clients in sync in our distributed data structures:

(Disclosure: Work at Microsoft, but I work in Azure and some open source stuff, not on or directly with Fluid/Office/etc.)

That's just a trademark clause for Microsoft logos and brands. The Fluid Framework itself is [MIT licensed](https://github.com/microsoft/FluidFramework/blob/main/LICENS...) and doesn't require exposing any of those logos/brands when you use it, so the framework itself is fairly open for usage.

I think the main thing that would slow down adoption for Fluid is that the only "production" backend is an Azure service, which isn't part of the open source Fluid Framework. [Other open source backends](https://fluidframework.com/docs/deployment/service-options/) aren't recommended for productions. Until there are some open source ones, I'd assume adoption will be limited to folks in the Azure ecosystem.

Within the context of the Microsoft-verse, Fluid Framework (https://fluidframework.com) is supposed to be solving similar problems in web apps, although I haven't personally played with it.

Microsoft recently open sourced fluid framework. It is a distributed, consensus based, real time collaboration framework written in typescript. Fluid would keep your clients synced up and your server code would only have to handle when someone hits submit. Fluid Framework

The official documentation and the github repository seem clear.