xnvme
ScaleStore
xnvme | ScaleStore | |
---|---|---|
3 | 3 | |
253 | 120 | |
3.2% | 0.8% | |
9.0 | 3.5 | |
3 days ago | 4 months ago | |
C | C++ | |
GNU General Public License v3.0 or later | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
xnvme
- Ask HN: Why are there no open source NVMe-native key value stores in 2023?
-
Anyone using io_uring?
Basically io_uring grew a lot, the latest API offers incredible tools but there are very few examples in any language (xnvme and fio), and in particular I'm struggling to understand how to do it in Rust: where should unsafe code stop? should I simply expose the io_uring api as unsafe, or should I do more work in C and present the ring when ready to rust?
-
libnvme VS xNVMe - a user suggested alternative
2 projects | 4 Mar 2022
The foundation of xNVMe is libxnvme, a user space library for working with NVMe devices. It provides a C API for memory management, that is, for allocating physical / DMA transferable memory when needed. An NVMe command interface allowing you to submit and complete NVMe commands in a synchronous as well as an asynchronous manner. Similar to libnvme, however, with a focus on I/O performance and portability. Thus, xNVMe and the libxnvme library works not just on Linux, but also on FreeBSD, Windows, and MacOS.
ScaleStore
-
Show HN: EloqKV – Scalable Distributed Acid Key-Value Database with Redis API
That is really interesting. It indeed reminds me a bit on this research project https://github.com/DataManagementLab/ScaleStore (the one Andy posted the analysis)
-
Ask HN: Why are there no open source NVMe-native key value stores in 2023?
I don't remember exactly why I have any of them saved, but these are some experimental data stores that seems to be fitting what you're looking for somewhat:
- https://github.com/DataManagementLab/ScaleStore - "A Fast and Cost-Efficient Storage Engine using DRAM, NVMe, and RDMA"
- https://github.com/unum-cloud/udisk - "The fastest ACID-transactional persisted Key-Value store designed for NVMe block-devices with GPU-acceleration and SPDK to bypass the Linux kernel."
- https://github.com/capsuleman/ssd-nvme-database - "Columnar database on SSD NVMe"
-
The end of a myth: Distributed transactions can scale
The linked blog post at the top of this article - https://muratbuffalo.blogspot.com/2023/01/is-scalable-oltp-i... - provides graphics that give extremely useful context. And here's the repo for the paper that discusses: https://github.com/DataManagementLab/ScaleStore
The idea that one of many writer-compute-nodes can literally reach into a memory buffer that is shared across machines, atomically flip some lock bits and propagate some cache-coherence messages, and use that to build a multi-writer distributed database without needing to partition (and where any writer-compute-node can handle any message, so you can just round-robin a firehose of messages at them)... and that there's a chance (though not yet implemented) that one could implement ACID on top of this? It's absolute madness, and wildly exciting.
What are some alternatives?
VkFFT - Vulkan/CUDA/HIP/OpenCL/Level Zero/Metal Fast Fourier Transform library
solid_cache - A database-backed ActiveSupport::Cache::Store
fio - Flexible I/O Tester
KVRocks - RocksDB compatible key value store and MyRocks compatible storage engine designed for KV SSD
openSeaChest - Cross platform utilities useful for performing various operations on SATA, SAS, NVMe, and USB storage devices.
uNVMe - KV and LBA SSD userspace NVMe driver
CheekyKitten - The Cheeky Kitten encoder is a simple binary obsfucation tool. Corrupts binary using a hex shuffle algorithm.
KVSSD - KV SSD host software including APIs and drivers
ssd-nvme-database - Columnar database on SSD NVMe
libnvme - C Library for NVM Express on Linux
cppdataloader - cppdataloader is a batching and caching library for C++17