flatterer
zune-jpeg
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
flatterer
-
How to convert a massive JSON into CSV
This tool was built for this purpose https://github.com/kindly/flatterer
-
Show HN: Flatterer Lite, A Rust WASM JSON to tabular converter, for nested data
I have been working on this Rust based JSON flattener (https://github.com/kindly/flatterer) to XLSX/CSV/SQLITE/PARQUET/POSTGRES for about 2 years. I finally made a WASM based one for CSV/XLSX which is really easy to play with. You can use it via a url (if the data has CORS headers), for example https://lite.flatterer.dev?url=https%3A%2F%2Fdummyjson.com%2....
The main difference with other converters/flatteners is that for one-to-many relationships (lists of objects in JSON) it produces a multi table output with keys to help join the tables together. Essentially produces a relational schema, along with some type information, of the JSON provided. For simple non-nested JSON it works too, but is a bit overkill.
Its all opensource and free, and I think that it will become more and more useful as JSON/Graphql APIs proliferate.
-
Create table from JSON
flatterer is the easiest option and will do this for you. It will also load the data into a sqlite or postgres database for you.
-
Robust Rust Library for Converting JSON Objects into CSV Rows
I have also written something similar but higher level flatterer which is python but is a binding to libflatterer. Its main difference is when there is a list of objects it treats it a one to many and creates a separate CSV file.
-
Flatten a massive json file
flatterer (github) is designed for this.
-
Simple GUI tool to query JSON or CSV databases [I have no coding skills]
This looks great and using duckdb as a backend us a great choice. For JSON I suggest using flatterer https://flatterer.opendata.coop/ . Which will convert most JSON into CSV/Parquet with very little technical knowledge.
-
Rust for Data Engineering
My project flatterer (used for flattening JSON into tables) also is mainly a python cli/libarary but is written in Rust for speed/memory efficiency and better concurrency.
-
JSON Processing
I have been working on a similar tool for the last while called flatterer (github). It has a similar aim and works with most kinds of JSON well (but not very well with top level objects like yours).
-
Stream processing - Nested JSON to Parquet
You could try Option B with flatterer https://flatterer.opendata.coop/ which purpose is to convert nested JSON into other formats without a schema upfront including parquet.
-
Turning JSON arrays into tables - Data flattening
flatterer docs (github)
zune-jpeg
-
Announcing zune-jpeg: Rust's fastest JPEG decoder
Congrats, but you dropped this.
-
Do you ever use unsafe { .. } when not implementing custom data structures or interacting with external C code?
Alternatively, you can round up the array length to the nearest power of 2 and use cheap bitmasking instead of branching bounds checks. If it goes wrong, it will access the wrong element but will not result in any code execution vulnerabilities. Here's an example of this in zune-jpeg.
-
Good example of high performance Rust project without unsafe code?
gif, png, zune-jpeg are on par with their C counterparts in terms of performance
-
What's everyone working on this week (including AoC) 49/2021?
here
-
Where do people learn to write truly quick software?
Based on some experience writing a jpeg decoder library, I'd like to give my two cents worth of information
-
What are you using Rust for?
Answer pretty fast
What are some alternatives?
json2csv - command line tool to convert json to csv
image-shrinker-lite - Drag-and-drop image compression app.
libflatterer - Lib flatterer: A lib to make JSON flatterer
justrunmydebugger - just run my debugger. see package here: https://build.opensuse.org/package/show/home:ila.embsys:justrunmydebugger/justrunmydebugger
miller - Miller is like awk, sed, cut, join, and sort for name-indexed data such as CSV, TSV, and tabular JSON
shiba - Display a random Shiba from your terminal whenever you feel the need to. Because why not?
flatten-tool - Tools for generating CSV and other flat versions of the structured data
calligrapher-ai - Handwriting Synthesis with RNNs ✍🏻
jq - Command-line JSON processor [Moved to: https://github.com/jqlang/jq]
nvim-matrix-bot - Just a bot for Neovim's Matrix room(s)
json-roller - Command line tool for converting json data structures to tables
aoc2021 - advent of code 2021 solutions