TurboBench
Moby
TurboBench | Moby | |
---|---|---|
10 | 213 | |
312 | 67,768 | |
- | 0.3% | |
8.9 | 10.0 | |
9 months ago | 4 days ago | |
C | Go | |
- | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
TurboBench
-
Zstd Content-Encoding planned to ship with Chrome 123
I'm still unconvinced about this addition. And I don't even dislike Zstandard.
The main motivation seems to be that while Zstandard is worse than Brotli at the highest level, it's substantially faster than Brotli when data has to be compressed on the fly with a limited computation budget. That might be true, but I'm yet to see any concrete or even anecdotal evidence even in the issue tracker [1] while there exist some benchmarks where both Zstandard and Brotli are fast enough for the web usage even at lower levels [2].
According to their FAQ [3] Meta and Akamai have successfully used Zstandard in their internal network, but my gut feeling is that they never actually tried to optimize Brotli instead. In fact, Meta employs the main author of Zstandard so it would have been easier to tune Zstandard instead of Brotli. While Brotli has some fundamental difference from Zstandard (in particular Brotli doesn't use arithmetic-equivalent coding), no one has concretely demonstrated that difference would prevent Brotli from being fast enough for dynamic contents in my opinion.
[1] https://issues.chromium.org/issues/40196713
[2] https://github.com/powturbo/TurboBench/issues/43
[3] https://docs.google.com/document/d/14dbzMpsYPfkefAJos124uPrl...
- TurboBench: Dynamic/Static web content compression benchmark
-
Ebiggers/libdeflate: Heavily optimized DEFLATE/zlib/gzip library
libdeflate compress better and has faster decompression than igzip.
See the silesia single core in-memory benchmark here [1] comparing zlib,libdeflate,igzip,...
https://github.com/powturbo/TurboBench/issues/4
-
Intel QuickAssist Technology Zstandard Plugin for Zstandard
- https://github.com/powturbo/TurboBench/issues/43
[1] https://github.com/powturbo/TurboBench
-
Variation on RLE to Achieve Lossless Compression for Tabular Data
Compressesing your sample file, we get 823 bytes with brotli
Download TurboBench and make your own tests:
[1] - https://github.com/powturbo/TurboBench
-
Data Compression Drives the Internet. Here’s How It Works
- igzip 1,2 is best for very fast networks > 10MB/s
brotli bring little value at decompression for users
[1] https://github.com/powturbo/TurboBench
[1] https://sites.google.com/site/powturbo/home/web-compression
[2] https://encode.su/threads/2333-TurboBench-Back-to-the-future...
-
Pigz: Parallel gzip for modern multi-processor, multi-core machines
Build or download TurboBench [1] executables for linux and windows from releases [2] ans make your own tests comparing oodle,zstd and other compressors.
[1] https://github.com/powturbo/TurboBench
[2] https://github.com/powturbo/TurboBench/releases
Moby
- An open framework to assemble specialized container systems
-
Release Radar • March 2024 Edition
Having been featured in our February 2023, and January 2024 Release Radars, Moby is the original Linux Container runtime. This new version adds a bunch of changes to the Docker CLI and Moby itself with additional features. There's bug fixes and enhancements, with the main thing for users to be on the look out for containers that were created using Docker Engine 25.0.0. These containers might have duplicate MAC addresses, and thus must be recreated. The same goes for those containers created with Moby 25.0+ and with user defined MAC addresses. Read up on all these changes in the release notes.
-
Choosing a Name for Your Computer
Formlabs does this as well for their 3d printers, my earliest encounter of this was when Docker started getting popular: https://github.com/moby/moby/blob/master/pkg/namesgenerator/...
- Docker Inc. refuses to patch HIGH vulnerabilities in Docker
-
Do not install Docker Desktop on GNU/Linux systems
Try to use moby instead since that is the engine in Docker.
https://github.com/moby/moby
-
Exploring Podman: A More Secure Docker Alternative
> Podman is designed to help with this by providing stronger default security settings compared to Docker. Features like rootless containers, user namespaces, and seccomp profiles, while available in Docker, aren't enabled by default and often require extra setup.
Seccomp has been enabled by default since 2015: https://github.com/moby/moby/pull/18780
It is true that Rootless isn't enabled by default but its "extra setup" can be done with a single command (`dockerd-rootless-setuptool.sh install`)
- Moby: Block io_uring_* syscalls in default profile
- Io_uring will be blocked by default on Docker
-
OpenZFS 2.2: Block Cloning, Linux Containers, BLAKE3
Perhaps.
Thing is, https://github.com/moby/moby/blob/670bc0a46c4ca03b75f1e72f73... is using https://github.com/mistifyio/go-zfs which features code like `out, err := zfsOutput("get", "-H", key, d.Name)` (Source: https://github.com/mistifyio/go-zfs/blob/master/zfs.go#L315) to get a single zfs property.
Somebody chose to use a library as abstraction that looks good but is implemented as a MVP (nothing wrong with that). "In the future, we hope to work directly with libzfs" should have raised an alarm somewhere, though.
What are some alternatives?
QAT-ZSTD-Plugin
podman - Podman: A tool for managing OCI containers and pods.
rapidgzip - Gzip Decompression and Random Access for Modern Multi-Core Machines
containerd - An open and reliable container runtime
libdeflate - Heavily optimized library for DEFLATE/zlib/gzip compression and decompression
nerdctl - contaiNERD CTL - Docker-compatible CLI for containerd, with support for Compose, Rootless, eStargz, OCIcrypt, IPFS, ...
pigz - A parallel implementation of gzip for modern multi-processor, multi-core machines.
docker-openwrt - OpenWrt running in Docker
lib842
ofelia - A docker job scheduler (aka. crontab for docker)
DirectStorage - DirectStorage for Windows is an API that allows game developers to unlock the full potential of high speed NVMe drives for loading game assets.
k3d - Little helper to run CNCF's k3s in Docker