qsv
tsv-utils
qsv | tsv-utils | |
---|---|---|
13 | 10 | |
2,234 | 1,396 | |
- | 0.0% | |
9.9 | 0.0 | |
2 days ago | over 1 year ago | |
Rust | D | |
The Unlicense | Boost Software License 1.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
qsv
- FLaNK Weekly 31 December 2023
-
Qsv: Efficient CSV CLI Toolkit
Thanks for the detailed feedback @snidane!
As maintainer of qsv, here's my reply:
- Given qsv's rapid release cycle (173 releases over three years), the auto-update check is essential at the moment. Once we reach 1.0, I'll turn it off. For now, given your feedback, I've only made it check 10% of the time.
- Pivot is in the backlog and I'll be sure to add unpivot when I implement it. (https://github.com/jqnatividad/qsv/issues/799)
- I'll add a dedicated summing command with the group by (-by) and window by (-over) capability (https://github.com/jqnatividad/qsv/issues/1514). Do note that `stats` has basic sum as @ezequiel-garzon pointed out.
- With the `enum` command, qsv can achieve what you proposed with `laminate`. E.g. qsv enum --new-column newcol --constant newconstant mydata.csv --output laminated-data.csv
- With the cat rowskey command, qsv can already concatenate files with mismatched headers.
- other file formats. qsv supports parquet, csv, tsv, excel, ods, datapackage, sqlite and more (see https://github.com/jqnatividad/qsv/tree/master#file-formats). Fixed-format though is not supported yet and quite interesting, and have added it to the backlog (https://github.com/jqnatividad/qsv/issues/1515)
- as to "enable embedding outputs of commands", qsv is composable by design, so you can use standard stdin/stdout redirection/piping techniques to have it work with other CLI tools like jq, awk, etc.
Finally, just released v0.120.0 that already incorporates the less aggressive self-update check. https://github.com/jqnatividad/qsv/releases/tag/0.120.0
- Joining CSV Data Without SQL: An IP Geolocation Use Case
-
Why my favourite API is a zipfile on the European Central Bank's website
qsv [1] also has a sqlp command which lets you run Polars SQL queries (even on multiple files). Here I'll send the csv data from stdin (represented by -) and then (optionally) pipe the output to the table command for formatting. The shape of the result is also printed to stderr (the (4, 2) below).
[1] https://github.com/jqnatividad/qsv
$ echo 'Name,Department,Salary
- Qsv: Ultra-fast CSV data-wrangling toolkit
- Qsv: CSVs sliced, diced and analyzed (fork of xsv)
- Nushell.sh ls – where size > 10mb – –sort-by modified
-
Do Rust and Lua work well together?
It works quite well IMHO. Using the mlua crate, I’ve managed to integrate Luau as a very powerful data-wrangling DSL for qsv (https://github.com/jqnatividad/qsv)
-
How manipulate this CSV in Python?
Maybe this might be better done using this? https://github.com/jqnatividad/qsv
-
How to convert xslx to csv using Rust?
https://github.com/jqnatividad/qsv is another option.
tsv-utils
-
Frawk: An efficient Awk-like programming language. (2021)
If you need just csv/tsv parsing, you can also take a look at https://github.com/eBay/tsv-utils
-
Tracking SQLite Database Changes in Git
You might want to look at tsv-utils, or a similar project: https://github.com/eBay/tsv-utils
For the SQL part, but maybe a lot heavier, you can use one of the projects listed on this page: https://github.com/multiprocessio/dsq (No longer maintained, but has links to lots of other projects)
-
I feel like an idiot but… I need Excel help.
TSV is most often a better format than CSV. Localization, in particular, is a nightmare with CSV.
- Splitting CSV files at 3GB/s
-
Modernizing AWK, a 45-year old language, by adding CSV support
For anything down and dirty, what's wrong with -F'"'? For anything fancy there are plenty of things like the below.
eBay's TSV Utilities: Command line tools for large, tabular data files. Filtering, statistics, sampling, joins and more.
includes csv to tsv: https://github.com/eBay/tsv-utils
HT: https://simonwillison.net/
-
Dlang 2.098.0 released, now available on OpenBSD
As an example, eBay's tsv-utils took full advantage of the GC and performed better than existing programs that had been hand-optimized in C etc.
-
[OC]Tidy Viewer (tv) is a cross-platform csv pretty printer that uses column styling to maximize viewer enjoyment.
tsv-utils - Command line csv data manipulation toolkit. D
-
Changing Registry Key Value Based on Contents of TXT/CSV File
In the majority of cases you'll be better off with Tab Separated Values over Comma Separated Values. More info here.
-
Return 1 to N results from a large (19MM line) CSV
May well be overkill for your needs, but I'm a fan of tsv-utils It's fast and enormously flexible, and seems to me a "best of breed" toolset for data mining CSV files (that is what it was written for). https://github.com/eBay/tsv-utils
What are some alternatives?
miller - Miller is like awk, sed, cut, join, and sort for name-indexed data such as CSV, TSV, and tabular JSON
dextool - Suite of C/C++ tooling built on LLVM/Clang
calamine - A pure Rust Excel/OpenDocument SpreadSheets file reader: rust on metal sheets
structured-text-tools - A list of command-line tools for manipulating structured text data
fortune-sheet - A drop-in javascript spreadsheet library that provides rich features like Excel and Google Sheets
csvtk - A cross-platform, efficient and practical CSV/TSV toolkit in Golang
csvquote
q - Quick and dirty debugging output for tired programmers. ⛺
goawk - A POSIX-compliant AWK interpreter written in Go, with CSV support
ess-view-data - View data support for ESS
zsv - zsv+lib: tabular data swiss-army knife CLI + world's fastest (simd) CSV parser