parquet-wasm
rson
| parquet-wasm | rson | |
|---|---|---|
| 6 | 1 | |
| 464 | 21 | |
| - | - | |
| 9.0 | 0.0 | |
| 3 days ago | about 3 years ago | |
| Rust | Rust | |
| Apache License 2.0 | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
parquet-wasm
- FLaNK AI Weekly for 29 April 2024
- Parquet-WASM: Rust-based WebAssembly bindings to read and write Parquet data
-
Goodbye, Node.js Buffer
nodejs-polars is node-specific and uses native FFI. polars can be compiled to Wasm but doesn't yet have a js API out of the box.
As for the fastest way to serialize data to Pandas data to the browser, you should use Parquet; it's the fastest to write on the Python side and read on the JS side, while also being compressed. See https://github.com/kylebarron/parquet-wasm (full disclosure, I wrote this)
-
Rust 1.63.0
I'm building WebAssembly bindings to existing Rust libraries [0] and lower-dependency geospatial tools [1]. Rust makes it very easy to bind rust code to both WebAssembly and Python. And by avoiding some large C geospatial dependencies we can get reliable performance in both wasm and Python using the exact same codebase.
[0]: https://github.com/kylebarron/parquet-wasm
[1]: https://github.com/kylebarron/geopolars
- Why isn’t there a decent file format for tabular data?
-
Recommendations when publishing a WASM library
Looks to be a great resource. I've been working on a WASM implementation of reading and writing Apache Parquet [0] and it's been difficult being new to WASM to find the best way of distributing the WASM that works on Node and through bundlers like Webpack.
[0]: https://github.com/kylebarron/parquet-wasm
rson
-
Why isn’t there a decent file format for tabular data?
Hm I wasn't aware of RSON. https://github.com/rson-rs/rson
QSN isn't intended to be tied to Rust in any way (and isn't), while RSON says it uses the Serde data model.
This gets at an issue I have been having a hard time explaining, mentioned here:
http://www.oilshell.org/blog/2022/03/backlog-arch.html
That is, narrow waists are necessarily a COMPROMISE. JSON is a compromise, and Rust users will be equally unhappy as Lua or Erlang users. That is a feature and not a bug for something meant of interoperability. You are "stuck with" the lowest common denominator, but that's what enables interop.
I contrast "monoglot" serialization formats like Python pickle an Go .gob with language-independent formats like JSON, TSV, and HTML. The wisdom of JSON is that Crockford specified it independently of JavaScript.
But both are useful.
It's not clear if RSON is meant to be monoglot or polyglot, but it's a huge difference and it seems more monoglot. QSN on the other hand is definitely a polyglot design like JSON, despite being derived from Rust.
What are some alternatives?
datasette-stripe - A web SQL interface to your Stripe account using Datasette.
hsv5 - HTML5 Based Alternative to CSV, TSV, JSONL, etc
quickjs-emscripten - Safely execute untrusted Javascript in your Javascript, and execute synchronous code that uses async functions
ndjson-spec - Specification
transmitic - Encrypted, peer to peer, file transfer program :: https://discord.gg/tRT3J6T :: https://www.reddit.com/r/transmitic/ :: https://twitter.com/transmitic
odiff - The fastest pixel-by-pixel image visual difference tool in the world.
geopolars - Geospatial extensions for Polars
AwesomeCSV - 🕶️A curated list of awesome tools for dealing with CSV.
csvz - The hot new standard in open databases
nodejs-polars - nodejs front-end of polars
zero-to-production - Code for "Zero To Production In Rust", a book on API development using Rust.