db_tutorial VS bolt

I am trying to recreate work from https://cstack.github.io/db_tutorial/ Basically it is recreating very simple SQLite, loading data from a file at the start of the program, and storing data in memory when the program runs then saving data to a file when we close the program. I believe that my issues are connected to working in the Windows instead of the Linux. Right now I am at part 5 when we start saving our data to the file when we close our program.

Stoped open source contributions recently, will be working on Build your Own Text Editor and Let's Build a Simple Database probably after landing an internship or a job

Last I heard, micro controllers still extensively use C and C-like sibling languages (like The Arduino Language). Most system applications use UI kits like Qt or GTK, which are also C libraries. Most of the Linux kernel consists of C code. Both SQL and NoSQL database backend are written mostly in C, with C++ coming in second, and only a few written in Rust, Zig or Golang (if you're interested in creating your own simple database, this is a great tutorial). TensorFlow and Pytorch, the machine learning libraries used for creating ANN-based architectures, uses libtensorflow and libtorch behind the scene that uses a mix of C and C++ libraries, then there's also GPU stuff like CUDA and RoCM that uses a mix of Fortran, C and Julia. Vulkan is also written in C and is supported in C++. Almost all performant libraries in Ruby, Python, Rust or Julia uses C bindings.

Have you thought about writing a DB from scratch in C? You'll learn lots of cool stuff about memory and OS abstractions: cstack.github.io/db_tutorial/

There's an online tutorial started a while back: https://cstack.github.io/db_tutorial

This crate started out as just a way for me to learn how boltdb works, while learning Rust at the same time. But somehow people started finding and using it and seem to like the simple API, so I figured I might as well share it in case someone else finds it useful too. If you want to know more about my motivations and the history of this crate, you can read the release notes on version 0.8.0!

Some example of embeddable database could be genji, badger and boltdb

Designing Data Intensive applications- specifically chapter 3 and 4 which deal with strategies and algorithms for storing and encoding data to be stored on disk and their pros and cons.

Once you read that, I'll suggest reading the source of a simple embedded key-value database, I wouldn't bother with RDBMs as they are complex beasts and contain way more than you need. BoltDB is a good project to read the source of https://github.com/boltdb/bolt, the whole thing is <10k lines of code and is a full blown production grade system with ACID semantics so packs a lot in those 10k and isn't just merely a toy.

Bolt db and Bolt db's author post to go with it.

https://github.com/boltdb/bolt is a ACID B+ tree key-value store

For a single server, SQLite, or boltdb[0]

I've never had to scale horizontally. I develop in Go and you can get very far along with just vertical scaling (aka beefier hardware).

Therefore I can't give concrete examples of a distributed db-as-a-library.

But all that you need is to extend the functions that fetch data to not just fetch from disk but from "peers" as well. For this to work you need servers (instances) to know about each other, and as you add more they also get added to their peers - sort of like a bittorrent network. I don't think it's difficult to do.

SQLite might not be suited for being distributed (although RQlite[1] claims to have done it).

Making a distributed data storage based on boltdb[0] is probably more feasible.

Whatever the case, there's no reason why a data storage engine can't be a library, even if it's distributed.

[0]: https://github.com/boltdb/bolt

[1]: https://github.com/rqlite/rqlite