Consider Using CSV

• tad

  3 3,013 7.6 TypeScript

  A desktop application for viewing and analyzing tabular data

Since this is about CSV, this is obligatory tool for larger ones:

* https://github.com/antonycourtney/tad

• KeenWrite

  98 621 0.0 Java

  Discontinued Free, open-source, cross-platform desktop Markdown text editor with live preview, string interpolation, and math.

• SurveyJS

  surveyjs.io sponsored

  Open-Source JSON Form Builder to Create Dynamic Forms Right in Your App. With SurveyJS form UI libraries, you can build and style forms in a fully-integrated drag & drop form builder, render them in your JS app, and store form submission data in any backend, inc. PHP, ASP.NET Core, and Node.js.

  

• parquet-go

  3 1,164 4.2 Go

  pure golang library for reading/writing parquet file

> It's so complex to work with, that unless you're specifically in data science, it's both unheard of and unusable.

FWIW, in my experience at a "data analytics platform" company, it's reasonably popular for data-heavy workflows since Because Parquet is well-defined, and file sizes are a fraction of their CSV equivalents.

> Is it a limitation of the format itself?

I don't think so. In other languages, you can generally read/write Parquet files without a ton of dependencies (e.g. https://github.com/xitongsys/parquet-go).

• js-bson

  1 1,093 8.1 TypeScript

  BSON Parser for node and browser

  

• ndjson.github.io

  17 23 0.0 CSS

  Info Website for NDJSON

  

No one uses that format for streamed json, see ndson and jsonl

http://ndjson.org/

The size complaint is overblown, as repeated fields are compressed away.

As other folks rightfully commented, csv is a mine field. One should assume every CSV file is broken in some way. They also don't enumerate any of the downsides of CSV.

What people should consider is using formats like Avro or Parquet that carry their schema with them so the data can be loaded and analyzed without have to manually deal with column meaning.

• xsv

  64 10,080 0.0 Rust

  A fast CSV command line toolkit written in Rust.

For manipulating CSV from the terminal, check out https://github.com/BurntSushi/xsv

• bsv

  6 31 3.7 C

  maximum performance data processing (by nathants)

i had a lot of fun exploring the performance ceiling of csv and csv like formats. turns out binary encoding of size prefixed byte arrays is fast[1].

csv is just a sequence of 2d byte arrays. probably avoid if dealing with heterogeneous external data. possibly use if dealing with homogeneous internal data.

https://github.com/nathants/bsv

• InfluxDB

  www.influxdata.com sponsored

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

Suggest a related project