py-caskdb VS dokany

self plug: I wanted to learn how databases work internally, like how they store and retrieve data, build indexes, etc.

I built an educational KV store to teach someone to write a database from scratch. I have set up this project in TDD fashion with the tests. So, you start with simple functions, pass the tests, and the difficulty level goes up. There are hints if you get stuck. When all the tests pass, you would have written a persistent key-value store in the end.

link: https://github.com/avinassh/py-caskdb

link: https://github.com/avinassh/py-caskdb

If you like databases, you start exploring the internals and start writing one! There is no going back once you dig deep into storage internals, KV stores, and distributed systems.

plug: I made an educational project which can help you write a database in python, from scratch - https://github.com/avinassh/py-caskdb

Build your own disk based KV store

link - https://github.com/avinassh/py-caskdb

It's not really the same though. A Projected File System copies the files from the backing store to somewhere on the local file system when requested and then performs IO normally on the local files.

For an actual implementation of userspace filesystems on Windows see dokany: https://github.com/dokan-dev/dokany

Now that you are inside the Mirror folder with cmd, let’s start the Mirror function to attach a NAS or cloud folder as a local disk in order to get recognized by the BB client and be backed up. Write something like (read the dokany documentation for more details here): mirror.exe /r x:\ /l m , where x:\ should be the letter of the folder attached with Raidrive and m:\ the letter you want to assign to the new local disk.

I lead a project that included shipping a filesystem driver and a virtual disk on Windows.

What I did to learn the lower-level APIs, and perform initial testing on the driver, was write a "mirror" drive. The user-mode code pointed to a folder on disk, the driver made a virtual disk drive, and all reads and writes in the virtual disk drive went to the mirror folder.

On Windows, you can implement something like that using Dokany, Dokan, or Winfsp. On linux, there's the Fuse API. On Mac, there's MacFUSE.

Even if you don't do a "mirror" drive, understanding the callbacks that libraries like Dokany, Dokan, Winfsp, and Fuse do helps you understand how IO happens in the driver. Many IO methods provided in popular languages provide abstractions above what the OS does. (For example, the Windows kernel has no concept of the "Stream" that's in your C# program. The "Stream"'s Position property is purely a construct within the .Net framework.)

https://dokan-dev.github.io/

https://github.com/dokan-dev/dokany

https://osxfuse.github.io/

Another place to start is the OS's documentation itself. For example, you can start with Window's CreateFileA function. This typically is what gets called "under the hood" in most programming languages when you open or create a file: https://learn.microsoft.com/en-us/windows/win32/api/fileapi/...

You'll need this ressource installed in 1.X version to make it work : https://github.com/dokan-dev/dokany

Potentially no need to buy any hardware https://github.com/dokan-dev/dokany/wiki/Use-Mirror-example

Yeah, I had an issue and had to downgrade the Dokan Library. I downloaded from here.