xvc VS hyperhyperspace-core

SEEKING WORK | REMOTE | Istanbul | UTC Business Hours

ML / MLOps / Data Engineering

Looking for projects that I can contribute in machine learning, data management, data pipelines, and similar technologies.

These days, I'm building an MLOps tool to manage data and pipelines on top of Git, in Rust.

https://github.com/iesahin/xvc (https://docs.xvc.dev)

I'm experienced in libraries like Tensorflow and PyTorch, languages (Rust, Python, Go, Dart, C, ...) and Docker, AWS, GCP. I used Vue.js & Flask for web, and Flutter for mobile in the past but these are not my current focus now.

I'm also a technical writer, and studying copywriting and marketing these days.

I can also architect your software.

https://emresahin.net/cv/ (I'm in the middle of updating the site but gives you an idea.)

[email protected]

Working on a new MLOps tool to manage files, data, pipelines, experiments and models.

I'm writing it in Rust. Using ECS instead of OOP. Going well so far.

It's open (and alpha.) https://github.com/iesahin/xvc

Documentation: https://docs.xvc.dev

Not all CRDT libraries focus on text editing. For example, I'm working on a Byzantine fault tolerant general-purpose data sync library loosely based on CRDTs: https://www.hyperhyperspace.org

I'm finding it painfully difficult but it is evolving steadily.

I'm thinking there's an interesting parallel between my browser-based p2p project [1] and cloudflare workers / DurableObjects. Instead of DurableObjects, we got HashedObjects [2], and instead of workers running on an edge network somewhere, we got in-browser p2p nodes running a browser-to-browser mesh network.

[1] Hyper Hyper Space: https://www.hyperhyperspace.org

I agree with the author on the merits of the file abstraction, but I think the concept should be updated for networked devices. We need file formats that support both offline usage and seamless sync over the network.

For example, here I use a merkle DAG-based file format to represent CRDT-like types:

https://www.hyperhyperspace.org

The resulting abstraction can be universally looked up using a hash (or short sequence of words), can be modified offline and synchronized flawlessly. It's still WIP (for example, you still can't export it to an actual file, hehe).

> I've been kicking around the idea of writing a CRDT-based editor using this model.

I got around to creating a data layer (p2p, browser-based, CRDT-backed) for something like this:

https://github.com/hyperhyperspace/hyperhyperspace-core

I'd be interested in collaborating on your editor

I'm working on it, Ben [1].

Believe me, it is not easy thing to do.

[1] https://github.com/hyperhyperspace/hyperhyperspace-core

Look at Hyper Hyper Space!

https://github.com/hyperhyperspace/hyperhyperspace-core

Its goals are similar, the approach is more pragmatic (p2p data layer using standard web browsers and webrtc).