ScaleStore
xNVMe
ScaleStore | xNVMe | |
---|---|---|
2 | 3 | |
108 | 213 | |
3.7% | 2.8% | |
3.4 | 9.3 | |
3 months ago | 2 days ago | |
C++ | C | |
MIT License | GNU General Public License v3.0 or later |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
ScaleStore
-
Ask HN: Why are there no open source NVMe-native key value stores in 2023?
I don't remember exactly why I have any of them saved, but these are some experimental data stores that seems to be fitting what you're looking for somewhat:
- https://github.com/DataManagementLab/ScaleStore - "A Fast and Cost-Efficient Storage Engine using DRAM, NVMe, and RDMA"
- https://github.com/unum-cloud/udisk - "The fastest ACID-transactional persisted Key-Value store designed for NVMe block-devices with GPU-acceleration and SPDK to bypass the Linux kernel."
- https://github.com/capsuleman/ssd-nvme-database - "Columnar database on SSD NVMe"
-
The end of a myth: Distributed transactions can scale
The linked blog post at the top of this article - https://muratbuffalo.blogspot.com/2023/01/is-scalable-oltp-i... - provides graphics that give extremely useful context. And here's the repo for the paper that discusses: https://github.com/DataManagementLab/ScaleStore
The idea that one of many writer-compute-nodes can literally reach into a memory buffer that is shared across machines, atomically flip some lock bits and propagate some cache-coherence messages, and use that to build a multi-writer distributed database without needing to partition (and where any writer-compute-node can handle any message, so you can just round-robin a firehose of messages at them)... and that there's a chance (though not yet implemented) that one could implement ACID on top of this? It's absolute madness, and wildly exciting.
xNVMe
- Ask HN: Why are there no open source NVMe-native key value stores in 2023?
-
Anyone using io_uring?
Basically io_uring grew a lot, the latest API offers incredible tools but there are very few examples in any language (xnvme and fio), and in particular I'm struggling to understand how to do it in Rust: where should unsafe code stop? should I simply expose the io_uring api as unsafe, or should I do more work in C and present the ring when ready to rust?
-
libnvme VS xNVMe - a user suggested alternative
2 projects | 4 Mar 2022
The foundation of xNVMe is libxnvme, a user space library for working with NVMe devices. It provides a C API for memory management, that is, for allocating physical / DMA transferable memory when needed. An NVMe command interface allowing you to submit and complete NVMe commands in a synchronous as well as an asynchronous manner. Similar to libnvme, however, with a focus on I/O performance and portability. Thus, xNVMe and the libxnvme library works not just on Linux, but also on FreeBSD, Windows, and MacOS.
What are some alternatives?
solid_cache - A database-backed ActiveSupport::Cache::Store
libnvme - C Library for NVM Express on Linux
ssd-nvme-database - Columnar database on SSD NVMe
VkFFT - Vulkan/CUDA/HIP/OpenCL/Level Zero/Metal Fast Fourier Transform library
KVRocks - RocksDB compatible key value store and MyRocks compatible storage engine designed for KV SSD
FEMU - FEMU: Accurate, Scalable and Extensible NVMe SSD Emulator (FAST'18). Please checkout https://github.com/vtess/FEMU for latest developments.
Olric - Distributed in-memory object store. It can be used as an embedded Go library and a language-independent service.
openSeaChest - Cross platform utilities useful for performing various operations on SATA, SAS, NVMe, and USB storage devices.
uNVMe - KV and LBA SSD userspace NVMe driver
fio - Flexible I/O Tester
KVSSD - KV SSD host software including APIs and drivers