zstd-proxy
sqlite-zstd
| zstd-proxy | sqlite-zstd | |
|---|---|---|
| 1 | 9 | |
| 73 | 1,349 | |
| - | - | |
| 10.0 | 2.2 | |
| over 1 year ago | 4 months ago | |
| C | Rust | |
| - | GNU Lesser General Public License v3.0 only |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
zstd-proxy
-
zstd
I needed to compress a TCP stream [0] for a block-storage project and I've been very impressed with zstd. Combined with io_uring, I'm getting less CPU usage and higher throughput to send and compress data than using socat to send uncompressed data.
[0]: https://github.com/fathyb/zstd-proxy
sqlite-zstd
-
WAL Mode in LiteFS
Have you compared LZ4 to other compression algorithms, zstd for example? ( https://github.com/phiresky/sqlite-zstd )
Given that LiteFS operates at the filesystem layer via FUSE, have you considered it against designs that use built-in features native to some filesystems? For example, I've considered a similar system design based on a single ZFS primary node that streams ZFS snapshots to reader nodes. With some coordination service (e.g. consul) it could still allow for the whole node promotion process.
- zstd
-
Compressed Sqlite database and indexing
There's library to compress sqlite databases with extensions like sqlite-zstd.
- Show HN: 通过透明压缩将SQLite数据库的大小减少到80%。 (Show HN: Reduce SQLite database size by up to 80% with transparent compression)
- Show HN: Reduce SQLite database size by up to 80% with transparent compression
-
SQLite Archive Files
You can also try sqlite-zstd [1], which is an sqlite extension allows transparent compression of individual rows by automatically training dictionaries based on groups of data rows.
Disclaimer: I made it and it's not production ready
[1] https://github.com/phiresky/sqlite-zstd
What are some alternatives?
libsqlfs - a library that implements a POSIX style filesystem on top of an SQLite database
compress - Optimized Go Compression Packages
pocketbase-framework-litestr
cloudbeaver - Cloud Database Manager
compress-json - Store JSON data in space efficient manner
postlite - Postgres wire compatible SQLite proxy.
pocketbase-framework-litestream - A small example using PocketBase as a Go Framework with replication/restoration of SQLite DB with Litestream
sqlite-web - Web-based SQLite database browser written in Python
LZ4 - Extremely Fast Compression algorithm
litexplore - Remote SQLite database explorer
litefs - FUSE-based file system for replicating SQLite databases across a cluster of machines
zstd - Zstandard - Fast real-time compression algorithm