crossbow reviews and mentions

• Lightweight DataFrame for Scala: Release of Crossbow 0.1. ?
  1 project | /r/scala | 3 Oct 2021

  Github link here: https://github.com/audienceproject/crossbow Version 0.1.4 on Maven here: https://search.maven.org/artifact/com.audienceproject/crossbow_2.13/0.1.4/jar The library is a DataFrame implementation for "small-scale" analytics. It is inspired in the Spark DataFrame API, but intended for use cases where it would be overkill to run distributed map-reduce. My rule of thumb is somewhere up to the 100s of MB volume of data, i.e. something you can easily load into memory on a single machine. It has no external dependencies other than the Scala reflection library, meaning that it is very easy to include in a project and deploy to serverless containers such as AWS Lambda. Operations cannot be typechecked at compile time, but the fundamental design principle is to fail fast at runtime with explanatory error messages if the API has been misused. The internal data format is a columnar array-based structure, intended to strike a good balance between performance and freedom of use. All operations are specialized on the primitive types Int, Long, Double and Boolean, to avoid unnecessary boxing. Currently supports mapping, filtering, sorting, joining, grouping, exploding and union. Columns and rows can be accessed and enumerated in a typed manner. I run production workloads at my company that has greatly benefited from the use of a DataFrame abstraction divorced from Spark, in this case for applications that are‍ simply not designed as map-reduce jobs, but where small-scale data aggregation, transformation and reduction are still done frequently.