Worries about tANS?

• pcodec

  19 245 8.8 Rust

  Lossless compressor and decompressor for numerical data using quantiles

For context: I'm creating an experimental successor to my library Quantile Compression, which does good compression for numerical sequences and has several users. I have a variable number of symbols which may be as high as 212 in some cases, but is ~26 in most cases. The data is typically 216 to 224 tokens long.

• ryg_rans

  1 285 10.0 C++

  Simple rANS encoder/decoder (arithmetic coding-ish entropy coder).

RANS uses compact table -- at the cost of speed. https://github.com/rygorous/ryg_rans Unfortunately, there are bugs in the repository, you should not use the code as is. It is better to rewrite the code -- it is quite simple when you understand it. Author's blog (http://fgiesen.wordpress.com/) contains a lot of details and is very interesting.

• InfluxDB

  www.influxdata.com sponsored

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• FiniteStateEntropy

  4 1,263 0.0 C

  New generation entropy codecs : Finite State Entropy and Huff0

tANS block based : FSE

• htscodecs

  2 26 7.7 C

  Custom compression for CRAM and others.

  

rANS SIMD block based : htscodecs

• Turbo-Range-Coder

  6 61 8.9 C

  TurboRC - Fastest Range Coder + Arithmetic Coding / Fastest Asymmetric Numeral Systems

Adaptive rANS included in : Turbo-Range-Coder including a benchmark app to test raw/text data with 8/16/32 bits integers or 32 bits floating points

• WorkOS

  workos.com sponsored

  The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

  

Suggest a related project