precomp-cpp
vdo
precomp-cpp | vdo | |
---|---|---|
14 | 9 | |
391 | 189 | |
- | 1.1% | |
0.0 | 2.9 | |
2 months ago | 5 days ago | |
C | C | |
Apache License 2.0 | GNU General Public License v3.0 only |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
precomp-cpp
-
Does epub compress well?
You should look at Precomp - an apparently abandoned, but awesomely working arhiver that is dedicated to compressing those ZIP-based formats, and compresses them twice-thrice as small as LZMA2 can.
- Any suggestion for compressing a large batch of photos and videos into a single zip?
- Compression for Documents
-
Putting 5,998,794 books on IPFS
You can also have a look at this packer. It compresses PDF and EPUB 2-3 times smaller than 7z at maximum settings, at half the speed. I keep all my books in it and never had a problem.
- .bin compression
-
libgen currently adds around 1,300 books a day
There is an opensource archiver called precomp that is designed specifically for e-documents and compresses them 2-3 times smaller than 7z in ultra mode. It works, and I keep my e-books in it. However, it seems nobody else does, and development has stopped due to lack of interest.
-
AWS switch from gzip to zstd – about 30% reduction in compressed S3 storage
There are tools like preflate [1] or precomp [2] that guarantees a bitwise identical reconstruction, of course modulo bugs.
[1] https://github.com/deus-libri/preflate
[2] https://github.com/schnaader/precomp-cpp/ (which internally makes use of preflate)
-
Which compression method for archiving OS ISOs?
You may use "precomp" in order to seek and uncompress those parts. Then you can use a better compression algorithm : http://schnaader.info/precomp.php
-
Is there any research as to what's the best way to compress each file type?
Precomp http://schnaader.info/precomp.php is a program which will seek compressed parts and decompress those parts in order to compress with another algorithm. This greatly improve compression ratio with the other algorithm.
-
How can I perfectly recreate a PNG losslessly compressed to JXL?
This may be of help. (a255854 and d649919)
vdo
- AWS switch from gzip to zstd – about 30% reduction in compressed S3 storage
-
"lvextend" freezes my Debian server
It is true that I have not had time to concentrate fully on this issue in the last days, and it is difficult for me to understand some of the subtleties of the subject (english is not my main language). I would like to thank you for your help and patience! I really appreciate it. FYI, I will continue to work on this issue and will update this topic. I've also asked the VDO development team to take a look at it and to tell me if they have any ideas, you can see that discussion here.
-
New LVM VDO logical volume inactive at startup, even with "--activate y" parameter
No problem, maybe someone else will read it and be able to clarify all of this. I also asked the question on VDO Github repository.
-
Which directories should I move to a compressed/deduplicated volume ? Which should I not ?
Still working on this project, I can now compile and run LVM VDO under Debian, which is a great thing (I shared the script here).
-
Need help on BASH syntax
#!/bin/bash install_dependencies() { (( ${#} > 1 )) && { apt update apt install -y "$@" } || echo "No dependencies to install" } clean_dependencies() { (( ${#} > 1 )) && { apt autoremove --purge -y "$@" } || echo "No dependencies to remove" } clone_repo() { (( ${#} == 1 )) && { TEMPDIR=$(mktemp -d "${1/*\/}.XXX") git clone "$1" "${TEMPDIR}" && \ cd "$TEMPDIR" && pwd -P } } # First part : KVDO module install_dependencies git make "linux-headers-$(uname -r)" > /dev/null 2>&1 WORK_DIR="$(clone_repo https://github.com/tigerblue77/kvdo.git)" ( cd "${WORK_DIR}" && \ make -j"$(nproc)" -C "/usr/src/linux-headers-$(uname -r)" M="${WORK_DIR}" ) > kvdo_log 2>&1 cp "${WORK_DIR}/vdo/kvdo.ko" "${WORK_DIR}/uds/uds.ko" "/lib/modules/$(uname -r)" clean_dependencies git make "linux-headers-$(uname -r)" > /dev/null 2>&1 rm -Rf "${WORK_DIR}" # Second Part: VDO Module install_dependencies git make gcc uuid-dev libz-dev libdevmapper-dev libblkid-dev > /dev/null 2>&1 WORK_DIR="$(clone_repo https://github.com/dm-vdo/vdo.git)" ( cd "${WORK_DIR}" && \ make -j"$(nproc)" && \ make -j"$(nproc)" install ) > vdo_build_log 2>&1 clean_dependencies git make gcc uuid-dev libz-dev libdevmapper-dev libblkid-dev > /dev/null 2>&1 rm -Rf "${WORK_DIR}" # Last Part depmod modprobe kvdo modprobe uds
-
Unable to convert /home LVM logical volume to LVM VDO volume
Update : the project is progressing and I managed to install VDO on Debian 11(.3) : https://github.com/dm-vdo/vdo/issues/47
What are some alternatives?
PDF-Writer - High performance library for creating, modiyfing and parsing PDF files in C++
ansitest - ansible test stuff and root/bin bash scripts for Linux / OSX admins
clawPDF - Open Source Virtual (Network) Printer for Windows that allows you to create PDFs, OCR text, and print images, with advanced features usually available only in enterprise solutions.
kvdo - A kernel module which provide a pool of deduplicated and/or compressed block storage.
p7zip - A new p7zip fork with additional codecs and improvements (forked from https://sourceforge.net/projects/sevenzip/ AND https://sourceforge.net/projects/p7zip/).
preflate - deflate stream reconstruction library
mcm - MCM file compressor
kvdo - A pair of kernel modules which provide pools of deduplicated and/or compressed block storage.
xdelta - open-source binary diff, delta/differential compression tools, VCDIFF/RFC 3284 delta compression
minio - The Object Store for AI Data Infrastructure
qpdf-rs - Rust bindings for QPDF C++ library
zstd - Zstandard - Fast real-time compression algorithm