Parquet-WASM: Rust-based WebAssembly bindings to read and write Parquet data

1. parquet-wasm

   1 6 584 7.9 Rust

   Rust-based WebAssembly bindings to read and write Apache Parquet data

   

2. SurveyJS

   surveyjs.io featured

   JavaScript Form Builder with No-Code UI & Built-In JSON Schema Editor. Add the SurveyJS white-label form builder to your JavaScript app (React/Angular/Vue3). Build complex JSON forms without coding. Fully customizable, works with any backend, perfect for data-heavy apps. Learn more.

   

3. duckdb-wasm

   2 19 1,586 9.4 TypeScript

   WebAssembly version of DuckDB

   

   i think duckdb-wasm is closer to 6MB over wire, but ~36MB once decompressed. (see net panel when loading https://shell.duckdb.org/)

   the decompressed size should be okay since it's not the same as parsing and JITing 36MB of JS.

4. lonboard

   3 3 752 8.7 Python

   A Python library for fast, interactive geospatial vector data visualization in Jupyter.

   

   I'll let Kyle chime in but I tested it a few months ago with millions of polygons on an M2 16GB of RAM laptop and it worked very well.

   There is a library by the same author called lonboard that provides the JS bits inside JupyterLab. https://github.com/developmentseed/lonboard

   I think it is based on the Kepler.gl / Deck.gl data loaders that go straight to GPU from network.

5. arrow-js-ffi

   4 1 117 7.5 TypeScript

   Zero-copy reading of Arrow data from WebAssembly

   

   Arrow JS is just ArrayBuffers underneath. You do want to amortize some operations to avoid unnecessary conversions. I.e. Arrow JS stores strings as UTF-8, but native JS strings are UTF-16 I believe.

   Arrow is especially powerful across the WASM <--> JS boundary! In fact, I wrote a library to interpret Arrow from Wasm memory into JS without any copies [0]. (Motivating blog post [1])

   [0]: https://github.com/kylebarron/arrow-js-ffi

   [1]: https://observablehq.com/@kylebarron/zero-copy-apache-arrow-...

6. pbf

   5 4 826 7.3 JavaScript

   A low-level, lightweight protocol buffers implementation in JavaScript.

   

   it's been about 3 years, but in Grafana at the time we were using something like ArrowJS + Arrow Flight + protobuf.js and then render the datasets into dashboards on Canvas, especially for streaming at ~20hz.

   when i benchmarked the fastest lib to simply convert the protobuf decode (https://github.com/mapbox/pbf), it was 5x slower than native JSON parsing in browsers for dataframe-like structures (e.g. a few dozen 2k-long arrays of floats).

7. InfluxDB

   www.influxdata.com featured

   InfluxDB – Built for High-Performance Time Series Workloads. InfluxDB 3 OSS is now GA. Transform, enrich, and act on time series data directly in the database. Automate critical tasks and eliminate the need to move data externally. Download now.

   

Suggest a related project