pigz
htslib
| pigz | htslib | |
|---|---|---|
| 9 | 7 | |
| 2,558 | 776 | |
| - | 1.7% | |
| 3.0 | 8.9 | |
| about 1 month ago | 2 days ago | |
| C | C | |
| - | GNU General Public License v3.0 or later |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
pigz
-
Pigz: Parallel gzip for modern multi-processor, multi-core machines
You can grab the version from the solaris userland repo I linked and use it without me completing a homework assignment. Just grab the pigz-2.3.4 source then apply the patches from [1] in the proper order. Maybe some of them aren't needed for non-Solaris.
1. https://github.com/oracle/solaris-userland/tree/master/compo...
I thought I had opened a PR for that a long while ago, but it doesn't show up on github these days. In any case, I did ask Mark Adler to review it. It was never a priority, then the code changed in ways that I don't really want to deal with.
While looking through the PRs, I noticed a PR for Blocked GZip Format (BGZF) [2]. That's very interesting, and perhaps suggests that bgzip is a tool you would be interested in.
2. https://github.com/madler/pigz/pull/19
- ZSTD 1.5.5 is released with a corruption fix found at Google
- pigz: A parallel implementation of gzip for multi-core machines
-
Pigz: A parallel implementation of gzip for multi-core machines
The bit I found most interesting was actually:
https://github.com/madler/pigz/blob/master/try.h
https://github.com/madler/pigz/blob/master/try.c
which implements try/catch for C99.
-
Is there stronger zip compression than 7z a -mx9?
pigz seems to be able to do it. https://github.com/madler/pigz
htslib
- Gentoo -Os vs -O3 application startup time?
- Does anyone know of a repository for actual genetic data?
-
Pigz: A parallel implementation of gzip for multi-core machines
There is another nice multi-core gzip based library called BGZF[1]. It is commonly used in bioinformatics. BGZF has the added advantage that it is block compressed with built in indexing method to permit seeking in compressed files.
[1] https://github.com/samtools/htslib
-
Tips for scalable workflows on AWS
In contrast, processing can start immediately and only transfer what is necessary if tooling can read bytes of data directly from Amazon S3. Tools based on htslib can do this, so you can run something like:
-
Software Development Project
Another idea is add in reading cloud data natively e.g. htslib which samtools relies on can read s3 directly. https://github.com/samtools/htslib
-
Interested in Bioinformatics / C pair-programming opportunity / learning / portfolio project.
Greetings! I'm a bioinformatics software dev in San Francisco and I'm looking for others interested in working on a high-performance genetic data analysis project. The project is in C, using https://github.com/samtools/htslib.
-
ffi-bitfield
I'm working on a bioinformatics-related binding called ruby-htslib. htslib makes heavy use of bit fields throughout the library, so supporting bit fields is inevitable.
What are some alternatives?
rapidgzip - Gzip Decompression and Random Access for Modern Multi-Core Machines
genozip - A modern compressor for genomic files (FASTQ, SAM/BAM/CRAM, VCF, FASTA, GFF/GTF/GVF, 23andMe...), up to 5x better than gzip and faster too
zstd
seqtk - Toolkit for processing sequences in FASTA/Q formats
mpifileutils - File utilities designed for scalability and performance.
bwa-mem2 - The next version of bwa-mem
TurboBench - Compression Benchmark
cyvcf2 - cython + htslib == fast VCF and BCF processing
isa-l - Intelligent Storage Acceleration Library
aws-genomics-workflows - Genomics Workflows on AWS
zstd - Zstandard - Fast real-time compression algorithm
libdna - ♥ Essential Functions for DNA Manipulation