precomp-cpp
zstd
precomp-cpp | zstd | |
---|---|---|
14 | 108 | |
391 | 22,445 | |
- | 1.5% | |
0.0 | 9.7 | |
2 months ago | 7 days ago | |
C | C | |
Apache License 2.0 | GNU General Public License v3.0 or later |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
precomp-cpp
-
Does epub compress well?
You should look at Precomp - an apparently abandoned, but awesomely working arhiver that is dedicated to compressing those ZIP-based formats, and compresses them twice-thrice as small as LZMA2 can.
- Any suggestion for compressing a large batch of photos and videos into a single zip?
- Compression for Documents
-
Putting 5,998,794 books on IPFS
You can also have a look at this packer. It compresses PDF and EPUB 2-3 times smaller than 7z at maximum settings, at half the speed. I keep all my books in it and never had a problem.
- .bin compression
-
libgen currently adds around 1,300 books a day
There is an opensource archiver called precomp that is designed specifically for e-documents and compresses them 2-3 times smaller than 7z in ultra mode. It works, and I keep my e-books in it. However, it seems nobody else does, and development has stopped due to lack of interest.
-
AWS switch from gzip to zstd – about 30% reduction in compressed S3 storage
There are tools like preflate [1] or precomp [2] that guarantees a bitwise identical reconstruction, of course modulo bugs.
[1] https://github.com/deus-libri/preflate
[2] https://github.com/schnaader/precomp-cpp/ (which internally makes use of preflate)
-
Which compression method for archiving OS ISOs?
You may use "precomp" in order to seek and uncompress those parts. Then you can use a better compression algorithm : http://schnaader.info/precomp.php
-
Is there any research as to what's the best way to compress each file type?
Precomp http://schnaader.info/precomp.php is a program which will seek compressed parts and decompress those parts in order to compress with another algorithm. This greatly improve compression ratio with the other algorithm.
-
How can I perfectly recreate a PNG losslessly compressed to JXL?
This may be of help. (a255854 and d649919)
zstd
-
Drink Me: (Ab)Using a LLM to Compress Text
> Doesn't take large amount of GPU resources
This is an understatement, zstd dictionary compression and decompression are blazingly fast: https://github.com/facebook/zstd/blob/dev/README.md#the-case...
My real-world use case for this was JSON files in a particular schema, and the results were fantastic.
-
SQLite VFS for ZSTD seekable format
This VFS will read a sqlite file after it has been compressed using [zstd seekable format](https://github.com/facebook/zstd/blob/dev/contrib/seekable_f...). Built to support read-only databases for full-text search. Benchmarks are provided in README.
-
Chrome Feature: ZSTD Content-Encoding
Of course, you may get different results with another dataset.
gzip (zlib -6) [ratio=32%] [compr=35Mo/s] [dec=407Mo/s]
zstd (zstd -2) [ratio=32%] [compr=356Mo/s] [dec=1067Mo/s]
NB1: The default for zstd is -3, but the table only had -2. The difference is probably small. The range is 1-22 for zstd and 1-9 for gzip.
NB2: The default program for gzip (at least with Debian) is the executable from zlib. With my workflows, libdeflate-gzip iscompatible and noticably faster.
NB3: This benchmark is 2 years old. The latest releases of zstd are much better, see https://github.com/facebook/zstd/releases
For a high compression, according to this benchmark xz can do slightly better, if you're willing to pay a 10× penalty on decompression.
xz -9 [ratio=23%] [compr=2.6Mo/s] [dec=88Mo/s]
zstd -9 [ratio=23%] [compr=2.6Mo/s] [dec=88Mo/s]
- Zstandard v1.5.6 – Chrome Edition
-
Optimizating Rabin-Karp Hashing
Compression, synchronization and backup systems often use rolling hash to implement "content-defined chunking", an effective form of deduplication.
In optimized implementations, Rabin-Karp is likely to be the bottleneck. See for instance https://github.com/facebook/zstd/pull/2483 which replaces a Rabin-Karp variant by a >2x faster Gear-Hashing.
- Show HN: macOS-cross-compiler – Compile binaries for macOS on Linux
-
Cyberpunk 2077 dev release
Get the data https://publicdistst.blob.core.windows.net/data/root.tar.zst magnet:?xt=urn:btih:84931cd80409ba6331f2fcfbe64ba64d4381aec5&dn=root.tar.zst How to extract https://github.com/facebook/zstd Linux (debian): `sudo apt install zstd` ``` tar -I 'zstd -d -T0' -xvf root.tar.zst ```
-
Honey, I shrunk the NPM package · Jamie Magee
I've done that experiment with zstd before.
https://github.com/facebook/zstd/blob/dev/programs/zstd.1.md...
Not sure about brotli though.
-
How in the world should we unpack archive.org zst files on Windows?
If you want this functionality in zstd itself, check this out: https://github.com/facebook/zstd/pull/2349
- Release Zstandard v1.5.5 · facebook/zstd
What are some alternatives?
PDF-Writer - High performance library for creating, modiyfing and parsing PDF files in C++
LZ4 - Extremely Fast Compression algorithm
clawPDF - Open Source Virtual (Network) Printer for Windows that allows you to create PDFs, OCR text, and print images, with advanced features usually available only in enterprise solutions.
Snappy - A fast compressor/decompressor
p7zip - A new p7zip fork with additional codecs and improvements (forked from https://sourceforge.net/projects/sevenzip/ AND https://sourceforge.net/projects/p7zip/).
LZMA - (Unofficial) Git mirror of LZMA SDK releases
mcm - MCM file compressor
7-Zip-zstd - 7-Zip with support for Brotli, Fast-LZMA2, Lizard, LZ4, LZ5 and Zstandard
xdelta - open-source binary diff, delta/differential compression tools, VCDIFF/RFC 3284 delta compression
ZLib - A massively spiffy yet delicately unobtrusive compression library.
qpdf-rs - Rust bindings for QPDF C++ library
brotli - Brotli compression format