Dolt Is Git for Data

1. go-mysql-server

   1 23 2,400 9.8 Go

   A MySQL-compatible relational database with a storage agnostic query engine. Implemented in pure Go.

   

   a very cool project they also maintain is a MySQL server framework for arbitrary backends (in Go): https://github.com/dolthub/go-mysql-server

   You can define a "virtual" table (schema, how to retrieve rows/columns) and then a MySQL client can connect and execute arbitrary queries on your table (which could just be an API or other source)

2. CodeRabbit

   coderabbit.ai featured

   CodeRabbit: AI Code Reviews for Developers. Revolutionize your code reviews with AI. CodeRabbit offers PR summaries, code walkthroughs, 1-click suggestions, and AST-based analysis. Boost productivity and code quality across all major languages with each PR.

   

3. dolt

   2 96 18,382 10.0 Go

   Dolt – Git for Data

   

4. sirix

   3 44 1,139 8.1 Roff

   SirixDB is an an embeddable, bitemporal, append-only database system and event store, storing immutable lightweight snapshots. It keeps the full history of each resource. Every commit stores a space-efficient snapshot through structural sharing. It is log-structured and never overwrites data. SirixDB uses a novel page-level versioning approach.

   

   Basically my research project[1] I'm working on in my spare time is all about versioning and efficiently storing small sized revisions of the data as well as allowing sophisticated time travel queries for audits and analysis.

   Of course all secondary user-defined, typed indexes are also versioned.

   Basically the technical idea is to map a huge tree of index tries (with revisions as indexed leave pages at the top-level and a document index as well as secondary indexes on the second level) to an append-only file. To reduce write amplification and to reduce the size of each snapshot data pages are first compressed and second versioned through a sliding snapshot algorithm. Thus, Sirix does not simply do a copy on write per page. Instead it writes nodes, which have been changed in the current revision plus nodes which fall out of the sliding window (therefore it needs a fast random-read drive).

   [1] https://github.com/sirixdb/sirix

5. lakeFS

   4 48 4,577 9.8 Go

   lakeFS - Data version control for your data lake | Git for data

   

   Also in the same vein, check out https://lakefs.io/

Suggest a related project