|4 days ago||2 days ago|
|GNU General Public License v3.0 or later||GNU General Public License v3.0 or later|
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
Bzip3 – a better and stronger spiritual successor to bzip2
4 projects | news.ycombinator.com | 10 May 2022
If anyone just cares for speed instead of compression I’d recommend lz4 . I only recently started using it. Its speed is almost comparable to memcpy.
Activision says Warzone doesn’t support map rotation because of ‘crazy install sizes’ | VGC
1 project | reddit.com/r/Games | 28 Mar 2022
gonna go off-topic, but doesn't LZ4 exists? I know IOI Interactive uses it for HITMAN 3.
I just took a random screenshot and made it look prettier. [ I don't know if this counts as fanart ]
2 projects | reddit.com/r/PixelGun | 20 Dec 2021
E: Realtime compression (A good compression library like Zstandard can make a game less than half the size while taking a tiny amount of CPU power when loading stuff. I think thats a pretty worthwhile trade.) (ZSTD github) (LZ4 github)
What's the best way to compress strings?
6 projects | reddit.com/r/cpp | 29 Jul 2021
lz4 for maximum decompression speed, for data that is often read but rarely written
Exomizer & Graforth
1 project | reddit.com/r/apple2 | 11 Jul 2021
That's great news. I implemented LZ4 for the Apple II, but Yann Collet is the creator of the algorithm.
How to become a tools/graphics/engine programmer
2 projects | reddit.com/r/gameenginedevs | 23 Apr 2021
Getting lost in material models is tempting. But, at this point you are overdue for working on your own asset pipeline. glTF is great. But, you should learn how to do it yourself. The hardest part will be reading source asset files. The FBX SDK is painful. Assimp isn't great either. Writing your own exporter to your own intermediate text format from Maya or Blender would be good if you are up for it. From whatever source, make your own archive format and binary formats for meshes, animations, textures and scenes. Use https://github.com/lz4/lz4 for compression. You should be able to decompress a list of assets into a big linear array and use them right there with just a bit of pointer fix-up. Minimize the amount of memory you have to touch from start to finish. Data that is going to the GPU (textures, vertex/index buffers) should decompress straight into mapped buffers for fast uploads.
Activision warns a standard 500GB PS4 may no longer fit Call of Duty: Warzone, Black Ops Cold War and Modern Warfare
1 project | reddit.com/r/Games | 25 Feb 2021
Oodle Kraken and lz4 are a thing.
LZ4, an Extremely Fast Compression Algorithm
9 projects | news.ycombinator.com | 27 Jan 2021
I'm not a fan of the stacked bar charts, I like the table of data for "Benchmarks" on the github source page: https://github.com/lz4/lz4
It makes it very clear where LZ4 fits into comparisons with compression speed, decompression speed and compression ratio
lz4_flex 0.7.2 reaches parity with cpp reference implementation on speed and ratio
3 projects | reddit.com/r/rust | 18 Jan 2021
It's hard to tell, how much since the latest lz4 1.9 has been improved, because the (benchmarks)[https://github.com/lz4/lz4/commit/31763c59fca9a38bc697cd37f591d6ba385f7614#diff-b335630551682c19a781afebcf4d07bf978fb1f8ac04c6bf87428ed5106870f5] are executed with different processors3 projects | reddit.com/r/rust | 18 Jan 2021
Which version are you using ? Is that the reference implementation at github.com/lz4/lz4 ?
RocksDB – Embeddable library for persistent key-value store with fast storage
1 project | news.ycombinator.com | 24 May 2022
In MongoDB you can have duplicate items even if you have unique index
2 projects | reddit.com/r/programming | 22 May 2022
But yeah, Facebook isn’t running on only MySQL, though they effectively built a graph database engine for it
Ask HN: Have you used SQLite as a primary database?
19 projects | news.ycombinator.com | 25 Apr 2022
Ask HN: What are the best key-value self-hosted storage engines?
5 projects | news.ycombinator.com | 31 Mar 2022
1 project | reddit.com/r/Database | 23 Mar 2022
How I install RocksDB
1 project | dev.to | 14 Mar 2022
#!/bin/bash wget https://github.com/facebook/rocksdb/archive/refs/tags/v6.15.5.tar.gz -O - \ | gzip -d \ | tar -xf - \ && pushd rocksdb-6.15.5 \ && cmake -DWITH_SNAPPY=YES -DCMAKE_INSTALL_PREFIX=$MTDIR . \ && DEBUG_LEVEL=0 PREFIX=$MTDIR make -j $(nproc) \ && DEBUG_LEVEL=0 PREFIX=$MTDIR make install \ && popd
Spring Kafka Streams playground with Kotlin - I
3 projects | dev.to | 9 Feb 2022
We will then create a GlobalKTable with a materialize view for the leverage-prices so we can use it to join with incoming quotes and expose the most up to date leverage for a stock taking advantage of the local default materialized state store Rocksdb that is created for us automatically by Kafka Streams.
Distributed SQL Essentials: Sharding and Partitioning in YugabyteDB
1 project | dev.to | 21 Nov 2021
The SST files store the key-value pairs for tables and indexes. Sharding is the right term here because each tablet is a database (based on RocksDB), with its own protection. This looks like the sharded databases we described above, except that they are not SQL databases but key-value document stores. They have all the required features for a reliable datastore, with transactions and strong consistency. However, they don’t have the burden of managing them as multiple databases because the SQL layer is above. Joins and secondary indexes are not processed at this level because this prevents cross-shard transactions.
Hello guys , needed help for building a key-value data store
1 project | reddit.com/r/Database | 9 Oct 2021
- RocksDB - kv store that uses LSM tree;
We built an open-source SQL DB for Intel SGX enclaves
3 projects | reddit.com/r/cybersecurity | 7 Aug 2021
Hi everyone! Our team just released EdgelessDB, an open-source database built on MariaDB that runs completely inside Intel SGX enclaves. As storage engine, it uses RocksDB with a custom encryption engine. The engine uses AES-GCM and is optimized for RocksDB’s specific SST file layout and the enclave environment. It has some nice properties like global confidentiality and verifiability and it considers strong attackers like malicious admins or rootkits. It also delivers rather low overheads (<10% for the TPC-C benchmark on Azure). In short: all data is only ever decrypted inside the enclave. This is different from other databases, where data and corresponding keys are processed in the clear in memory. We believe this is useful because (1) it’s very secure and (2) it enables some interesting use cases, like secure data pooling between parties. If you’re interested in trying it out: here’s a quickstart guide. In essence, you can run the Docker image with a single command on any recent Intel Xeon with SGX. Code and more info can be found on GitHub. Would be great to get your feedback on this :-)
What are some alternatives?
zstd - Zstandard - Fast real-time compression algorithm
brotli - Brotli compression format
Snappy - A fast compressor/decompressor
LZMA - (Unofficial) Git mirror of LZMA SDK releases
LMDB - Read-only mirror of official repo on openldap.org. Issues and pull requests here are ignored. Use OpenLDAP ITS for issues.
LevelDB - LevelDB is a fast key-value storage library written at Google that provides an ordered mapping from string keys to string values.
ZLib - A massively spiffy yet delicately unobtrusive compression library.
SQLite - Unofficial git mirror of SQLite sources (see link for build instructions)
sled - the champagne of beta embedded databases
ClickHouse - ClickHouse® is a free analytics DBMS for big data
skywater-pdk - Open source process design kit for usage with SkyWater Technology Foundry's 130nm node.
LZFSE - LZFSE compression library and command line tool