miller
fq
Our great sponsors
miller | fq | |
---|---|---|
63 | 43 | |
8,510 | 9,316 | |
- | - | |
9.1 | 9.4 | |
8 days ago | 1 day ago | |
Go | Go | |
GNU General Public License v3.0 or later | GNU General Public License v3.0 or later |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
miller
- Qsv: Efficient CSV CLI Toolkit
-
jq 1.7 Released
jq and miller[1] are essential parts of my toolbelt, right up there with awk and vim.
-
Perl first commit: a “replacement” for Awk and sed
> This works really well if your problem can be solved in one or two liners.
My personal comfort threshold is around the 100-line mark. It's even possible to write maintainable shell scripts up to 500 lines, but it mostly depends on the problem you're trying to solve, and the discipline of the programmer to follow best practices (use sane defaults, ShellCheck, etc.).
> It go bad very quickly when, say, you have two CSV files and want to join them the sql-way.
In that case we're talking about structured data, and, yeah, Perl or Python would be easier to work with. That said, depending on the complexity of the CSV, you can still go a long way with plain Bash with IFS/read(1) or tr(1) to split CSV columns. This wouldn't be very robust, but there are tools that handle CSV specifically[1], which can be composed in a shell script just fine.
So it's always a balancing act of being productive quickly with a shell script, or reaching out for a programming language once the tools aren't a good fit, or maintenance becomes an issue.
-
GQL: A new SQL like query language for .git files written in Rust
That said, you may be interested in Miller (https://github.com/johnkerl/miller) which provides similar capabilities for CSV, JSON, and XML files. It doesn't use a SQL grammar, but that's just the proverbial lipstick on the thing. I'm not the author, but I have used it and I see some parallels in use cases at the very least.
-
Any cli utility to create ascii/org mode tables?
worth giving Miller a shot
-
I wrote this iCalendar (.ics) command-line utility to turn common calendar exports into more broadly compatible CSV files.
CSV utilities (still haven't pick a favorite one...): https://github.com/harelba/q https://github.com/BurntSushi/xsv https://github.com/wireservice/csvkit https://github.com/johnkerl/miller
- Miller: Like Awk, sed, cut, join, and sort for CSV, TSV, and tabular JSON
- Nushell.sh ls – where size > 10mb – –sort-by modified
-
Tool to interact with CSV
Came here to recommend miller!
fq
-
Reverse-engineering an encrypted IoT protocol
Hey! fq author here. I have a bunch of related tools in the readme https://github.com/wader/fq?tab=readme-ov-file#tools two suggestions: gnu poke and wireshark (can decode lots of more things then just network protocol)
-
To a Man with `Jq`, Everything Looks Like JSON
Did someone say let's represent structured data as json? a bit of shameless plug: https://github.com/wader/fq :) It's using a fork of gojq btw!
-
Jaq – A jq clone focused on correctness, speed, and simplicity
https://github.com/wader/fq has a REPL and can read JSON. Tip is to use "paste | from_json | repl" in a REPl to paste JSON into a sub-REPL, you can also use `` with fq which is a raw string literal
Quite a lot! i use it to explode both JSON and tex (parse using jq functions). I also use it for exploring ane debug binary formats (https://github.com/wader/fq). Now a days i also use it for some adhoc programming and a calculator.
-
jq 1.7 Released
I do lots of exploratory work in various structure data, in my case often debugging media filea via https://github.com/wader/fq, which mean doing lots of use-once-queries on the command line or REPL. In those cases jq line-friendly and composable syntax and generators really shine.
- FLaNK Stack 5-June-2023
-
Fq: Jq for Binary Formats
Hi, i'm working on runtime kaitai support and have made good process but it's quite a big task. Keep an eye on https://github.com/wader/fq/issues/627 your interested.
Hi, sorry for the delay, on vacation. There is no process really more then convincing me :) and i think i will accept any decoder that is for a format used in public, standardized or proprietary.
I do want to add some kind of runtime format support and i'm working adding kaitai support but it's not ready yet, it's not an easy thing to do :) but i've made very good progress. ideally it will be something like: fq -d format.ksy file
Subscribe or keep an eye on this issue for updates https://github.com/wader/fq/issues/627
And feel free to ask any questions!
What are some alternatives?
visidata - A terminal spreadsheet multitool for discovering and arranging data
xsv - A fast CSV command line toolkit written in Rust.
jq - Command-line JSON processor [Moved to: https://github.com/jqlang/jq]
dasel - Select, put and delete data from JSON, TOML, YAML, XML and CSV files with a single tool. Supports conversion between formats and can be used as a Go package.
csvtk - A cross-platform, efficient and practical CSV/TSV toolkit in Golang
yq - yq is a portable command-line YAML, JSON, XML, CSV, TOML and properties processor
csvq - SQL-like query language for csv
json-toolkit - "the best opensource converter I've found across the Internet" -- dene14
gron - Make JSON greppable!
awesome-cli-apps - 🖥 📊 🕹 🛠 A curated list of command line apps
csvkit - A suite of utilities for converting to and working with CSV, the king of tabular file formats.