Manage all types of time series data in a single, purpose-built database. Run at any scale in any environment in the cloud, on-premises, or at the edge. Learn more →
Top 23 C++ Compression Projects
-
draco
Draco is a library for compressing and decompressing 3D geometric meshes and point clouds. It is intended to improve the storage and transmission of 3D graphics.
-
Project mention: Why I enjoy using the Nim programming language at Reddit. | /r/RedditEng | 2022-11-14
Another example of Nim being really fast is the supersnappy library. This library benchmarks faster than Google’s C or C++ Snappy implementation.
-
Mergify
Updating dependencies is time-consuming.. Solutions like Dependabot or Renovate update but don't merge dependencies. You need to do it manually while it could be fully automated! Add a Merge Queue to your workflow and stop caring about PR management & merging. Try Mergify for free.
-
1) I used meshoptimizer to simplify meshes and building meshlets. But to combine these meshlets (for more efficient simplifying) I built a graph with weights and partition it. In this weight some metrics are included: the shared border length of a pair of meshlets, their facing direction and other similar metrics. The next step is to simplify these groups of meshlets (fixate borders for each of them and simplify meshlets within the group). So, you can make dependencies of newly generated meshlets on the initial ones.
-
Project mention: Show HN: Want something better than k-means? Try BanditPAM | news.ycombinator.com | 2023-04-04
> frown on that sort of dataset
That example was definitely contrived and designed to strongly illustrate the point. I'll counter slightly that non-peaky topologies aren't uncommon, but they're unlikely to look anything that would push KMedoids to a pathological state rather than just a slightly worse state ("worse" assuming that KMeans is the right choice for a given problem).
> worth pointing out .. data reference
Totally agreed. I hope my answer didn't come across as too negative. It's good work, and everyone else was talking about the positives, so I just didn't want to waste too much time echoing again that while getting the other points across.
> bolt reference
https://github.com/dblalock/bolt
They say as much in their paper, but they aren't the first vector quantization library by any stretch. Their contributions are, roughly:
1. If you're careful selecting the right binning strategy then you can cancel out a meaningful amount of discretization error.
2. If you do that, you can afford to choose parameters that fit everything nicely into AVX2 machine words, turning 100s of branching instructions into 1-4 instructions.
3. Doing some real-world tests to show that (1-2) matter.
Last I checked their code wasn't very effective for the places I wanted to apply it, but the paper is pretty solid. I'd replace it with a faster KMeans approximation less likely to crash on big data (maybe even initializing with KMedoids :) ), and if the thing you're quantizing is trainable with some sort of gradient update step then you should do a few optimization passes in the discretized form as well.
-
caesium-image-compressor
Caesium is an image compression software that helps you store, send and share digital pictures, supporting JPG, PNG and WebP formats. You can quickly reduce the file size (and resolution, if you want) by preserving the overall quality of the image.
Project mention: Easy boxart resizing with imagemagick in your browser. No install, no CLI! | /r/RG35XX | 2023-05-05I also use Caesium Image Compressor on my ROMs and Themes folder to reduce their size and improve the RG35XX's responsiveness.
-
-
Project mention: I converted a massive library of mo-cap animations to .fbx which you can use freely with ALMOST no restrictions | /r/gamedev | 2022-11-28
For such animations make sure to check out ACL (animation compression library) by /u/zeno490. There are some stats and implementation details on how well it works on his blog: http://nfrechette.github.io/
-
SonarCloud
Analyze your C and C++ projects with just one click.. SonarCloud, a cloud-based static analysis tool for your CI/CD workflows, offers a one-click automatic analysis of C and C++ projects hosted on GitHub. Zero configuration and free for open-source projects! Analyze free.
-
Project mention: Help! Does anyone know how to install johncena141 games on linux? | /r/LinuxCrackSupport | 2023-07-01
on a fresh install all you need is dwarfs https://github.com/mhx/dwarfs and libopenal1
-
compressonator
Tool suite for Texture and 3D Model Compression, Optimization and Analysis using CPUs, GPUs and APUs
Project mention: I cannot open Skyrim's .dds files for editing using GIMP | /r/skyrimmods | 2023-01-28AMD’s Compressonator utility (https://github.com/GPUOpen-Tools/compressonator) is free and can provide much better image compression. It provides settings for adjusting the desired compression quality (better compression = slower). It can run either on CPU or GPU, though GPU will be faster, especially for BC7 compression. It works with my Nvidia card, so you don’t necessarily need an AMD card to use the utility, but YMMV depending on the GPU you’re trying to use.
-
Bareos
Bareos is a cross-network Open Source backup solution (licensed under AGPLv3) which preserves, archives, and recovers data from all major operating systems.
-
PGM-index
🏅State-of-the-art learned data structure that enables fast lookup, predecessor, range searches and updates in arrays of billions of items using orders of magnitude less space than traditional indexes
PGM Index
-
LZHAM
Lossless data compression codec with LZMA-like ratios but 1.5x-8x faster decompression speed, C/C++
-
clp
Compressed Log Processor (CLP) is a free tool capable of compressing text logs and searching the compressed logs without decompression. (by y-scope)
Project mention: FOSS, cloud native, log storage and query engine build with Apache Arrow & Parquet, written in Rust and React. | /r/rust | 2022-10-01Thoughts on integrating CLP with this infra? Not sure whether this even makes sense to try? LINK
-
-
SIMDCompressionAndIntersection
A C++ library to compress and intersect sorted lists of integers using SIMD instructions
-
GDCM
Grassroots DICOM read-only mirror. Only for Pull Request. Please report bug at http://sf.net/p/gdcm
-
-
-
zpaqfranz
Deduplicating archiver with encryption and paranoid-level tests. Swiss army knife for the serious backup and disaster recovery manager. Ransomware neutralizer. Win/Linux/Unix
Now, onto files backup - if you value your data, don't make just one backup copy, make two or three. Also, I'd recommend using software that will make snapshots and you could restore whichever version you need. I am using zpaqfranz for few years now, it is command line software but you can make batch file and update the archive when needed - it will add only new and changed files, so only first backup will last long.
-
-
Project mention: Libzim now has an official WebAssembly build target... why this is big (for us)! | /r/Kiwix | 2022-12-08
This is low-level stuff, but the new release of the backend that powers Kiwix clients, libzim 8.1.0 (changelog), now targets WebAssembly (a.k.a. WASM). This is big because it potentially brings the full powers of the Kiwix backend to Web apps such as Kiwix JS and Kiwix JS PWA/Electron. Up till now, these apps have relied on a custom JS port of parts of libzim, but it lacked full features, such as Full Text search. So, you will begin to see us integrating new features into the Web versions soon!
-
-
Squeezer (https://github.com/mzuther/Squeezer) Compressor and when set "correctly" it sounds really good.
-
InfluxDB
Collect and Analyze Billions of Data Points in Real Time. Manage all types of time series data in a single, purpose-built database. Run at any scale in any environment in the cloud, on-premises, or at the edge.
NOTE:
The open source projects on this list are ordered by number of github stars.
The number of mentions indicates repo mentiontions in the last 12 Months or
since we started tracking (Dec 2020).
The latest post mention was on 2023-07-01.
C++ Compression related posts
- BtrBlocks: Efficient Columnar Compression for Data Lakes [pdf]
- Help! Does anyone know how to install johncena141 games on linux?
- A.Total.War.Saga.THRONES.OF.BRITANNIA-TENOKE
-
entropy VS ent - a user suggested alternative
2 projects | 27 Jun 2023
- Nanite-like LODs experiments
- Will the same texture used in many gltfs be duplicated in memory?
- Maximum size of scene or a project in THREE.JS
-
A note from our sponsor - InfluxDB
www.influxdata.com | 26 Sep 2023
Index
What are some of the best open-source Compression projects in C++? This list will help you:
| Project | Stars | |
|---|---|---|
| 1 | draco | 5,874 |
| 2 | Snappy | 5,780 |
| 3 | meshoptimizer | 4,422 |
| 4 | bolt | 2,415 |
| 5 | caesium-image-compressor | 1,849 |
| 6 | dandere2x | 1,340 |
| 7 | acl | 1,159 |
| 8 | dwarfs | 1,158 |
| 9 | compressonator | 1,114 |
| 10 | Bareos | 873 |
| 11 | PGM-index | 704 |
| 12 | LZHAM | 671 |
| 13 | clp | 640 |
| 14 | EWAHBoolArray | 431 |
| 15 | SIMDCompressionAndIntersection | 398 |
| 16 | GDCM | 312 |
| 17 | CIDLib | 205 |
| 18 | grok | 173 |
| 19 | zpaqfranz | 172 |
| 20 | sevenzipjbinding | 150 |
| 21 | libzim | 146 |
| 22 | DeskX | 126 |
| 23 | Squeezer | 125 |
Collect and Analyze Billions of Data Points in Real Time
Manage all types of time series data in a single, purpose-built database. Run at any scale in any environment in the cloud, on-premises, or at the edge.
www.influxdata.com