Parquet-WASM: Rust-based WebAssembly bindings to read and write Parquet data

This page summarizes the projects mentioned and recommended in the original post on news.ycombinator.com

SurveyJS - Open-Source JSON Form Builder to Create Dynamic Forms Right in Your App
With SurveyJS form UI libraries, you can build and style forms in a fully-integrated drag & drop form builder, render them in your JS app, and store form submission data in any backend, inc. PHP, ASP.NET Core, and Node.js.
surveyjs.io
featured
InfluxDB - Power Real-Time Data Analytics at Scale
Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
www.influxdata.com
featured
  • parquet-wasm

    Rust-based WebAssembly bindings to read and write Apache Parquet data

  • SurveyJS

    Open-Source JSON Form Builder to Create Dynamic Forms Right in Your App. With SurveyJS form UI libraries, you can build and style forms in a fully-integrated drag & drop form builder, render them in your JS app, and store form submission data in any backend, inc. PHP, ASP.NET Core, and Node.js.

    SurveyJS logo
  • duckdb-wasm

    WebAssembly version of DuckDB

  • i think duckdb-wasm is closer to 6MB over wire, but ~36MB once decompressed. (see net panel when loading https://shell.duckdb.org/)

    the decompressed size should be okay since it's not the same as parsing and JITing 36MB of JS.

  • lonboard

    A Python library for fast, interactive geospatial vector data visualization in Jupyter.

  • I'll let Kyle chime in but I tested it a few months ago with millions of polygons on an M2 16GB of RAM laptop and it worked very well.

    There is a library by the same author called lonboard that provides the JS bits inside JupyterLab. https://github.com/developmentseed/lonboard

    I think it is based on the Kepler.gl / Deck.gl data loaders that go straight to GPU from network.

  • arrow-js-ffi

    Zero-copy reading of Arrow data from WebAssembly

  • Arrow JS is just ArrayBuffers underneath. You do want to amortize some operations to avoid unnecessary conversions. I.e. Arrow JS stores strings as UTF-8, but native JS strings are UTF-16 I believe.

    Arrow is especially powerful across the WASM <--> JS boundary! In fact, I wrote a library to interpret Arrow from Wasm memory into JS without any copies [0]. (Motivating blog post [1])

    [0]: https://github.com/kylebarron/arrow-js-ffi

    [1]: https://observablehq.com/@kylebarron/zero-copy-apache-arrow-...

  • pbf

    A low-level, lightweight protocol buffers implementation in JavaScript.

  • it's been about 3 years, but in Grafana at the time we were using something like ArrowJS + Arrow Flight + protobuf.js and then render the datasets into dashboards on Canvas, especially for streaming at ~20hz.

    when i benchmarked the fastest lib to simply convert the protobuf decode (https://github.com/mapbox/pbf), it was 5x slower than native JSON parsing in browsers for dataframe-like structures (e.g. a few dozen 2k-long arrays of floats).

  • InfluxDB

    Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

    InfluxDB logo
NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts

  • Waxolotl – a small language that compiles to WebAssembly text format

    3 projects | news.ycombinator.com | 18 May 2024
  • Prettier $20k Bounty was Claimed

    16 projects | news.ycombinator.com | 27 Nov 2023
  • Help achieving one culture

    1 project | /r/eu4 | 12 Oct 2023
  • Extism Makes WebAssembly Easy

    13 projects | news.ycombinator.com | 4 Oct 2023
  • Node on Web. Use Nodejs freely in your browser with Linux infrastructure.

    8 projects | /r/node | 3 Jul 2023