Pigz: A parallel implementation of gzip for multi-core machines

1. pigz

   1 10 2,737 0.0 C

   A parallel implementation of gzip for modern multi-processor, multi-core machines.

   

   The bit I found most interesting was actually:

   https://github.com/madler/pigz/blob/master/try.h

   https://github.com/madler/pigz/blob/master/try.c

   which implements try/catch for C99.

2. CodeRabbit

   coderabbit.ai featured

   CodeRabbit: AI Code Reviews for Developers. Revolutionize your code reviews with AI. CodeRabbit offers PR summaries, code walkthroughs, 1-click suggestions, and AST-based analysis. Boost productivity and code quality across all major languages with each PR.

   

3. htslib

   2 7 845 8.6 C

   C library for high-throughput sequencing data formats

   

   There is another nice multi-core gzip based library called BGZF[1]. It is commonly used in bioinformatics. BGZF has the added advantage that it is block compressed with built in indexing method to permit seeking in compressed files.

   [1] https://github.com/samtools/htslib

4. mpifileutils

   3 4 176 8.0 C

   File utilities designed for scalability and performance.

   

   If you ever run into the limitations of a single machine, dbz2 is also a fun little app for this sort of thing. You can run it across multiple machines and it'll automatically balance the workload across them.

   https://github.com/hpc/mpifileutils/blob/master/man/dbz2.1

5. PostgreSQL

   4 494 17,385 10.0 C

   Mirror of the official PostgreSQL GIT repository. Note that this is just a *mirror* - we don't work with pull requests on github. To contribute, please see https://wiki.postgresql.org/wiki/Submitting_a_Patch

   

   Postgres uses a similar custom implementation of try/catch [1].

   [1] https://github.com/postgres/postgres/blob/master/src/include...

Suggest a related project