binary-parsing
json-toolkit
| binary-parsing | json-toolkit | |
|---|---|---|
| 5 | 5 | |
| 839 | 67 | |
| - | - | |
| 5.7 | 4.6 | |
| 27 days ago | about 1 year ago | |
| Python | ||
| MIT License | GNU General Public License v3.0 only |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
binary-parsing
- Reverse-engineering an encrypted IoT protocol
-
GNU poke: The extensible editor for structured binary data
* binary-parsing - https://github.com/dloss/binary-parsing
-
Ask HN: What software do you use to examine binary files?
There are a few hex/disk editors that support "templates" (but you need most times to create those yourself).
Here is a sort of "curated list" of related tools:
https://github.com/dloss/binary-parsing
The most complete/populated I know of is Kaitai:
http://kaitai.io/
http://formats.kaitai.io/
that you can use with Hiew with Kiewtai
https://github.com/taviso/kiewtai
If the question is slightly different, i.e. which bytes are used to identify a given file format, there is Trid:
https://mark0.net/soft-trid-e.html
Which has also a database of known headers/patterns.
- A list of tools for parsing binary data structures
-
Fq: Jq for Binary Formats
Nice! Some other tools and parsers: https://github.com/dloss/binary-parsing
json-toolkit
-
Show HN: Comma Separated Values (CSV) to Unicode Separated Values (USV)
CSV is great because excel can import it, but it can't import USV, so at that point, why use USV when you can use JSON?
https://github.com/tyleradams/json-toolkit/
-
Analyzing multi-gigabyte JSON files locally
> Also note that this approach generalizes to other text-based formats. If you have 10 gigabyte of CSV, you can use Miller for processing. For binary formats, you could use fq if you can find a workable record separator.
You can also generalize it without learning a new minilanguage by using https://github.com/tyleradams/json-toolkit which converts csv/binary/whatever to/from json
- Fq: Jq for Binary Formats
-
Show HN: Angle Grinder – A terminal app to slice, dice, and aggregate your logs
I really like this tool, but I'm not sure what it gets me more than jq (and https://github.com/tyleradams/json-toolkit to convert non-json to json).
What can angle grinder do better than jq?
- Show HN: Transform a CSV into a JSON and vice versa
What are some alternatives?
HexFiend - A fast and clever hex editor for macOS
miller - Miller is like awk, sed, cut, join, and sort for name-indexed data such as CSV, TSV, and tabular JSON
fq - jq for binary formats - tool, language and decoders for working with binary and text formats
ndjson - Streaming line delimited json parser + serializer
Kaitai Struct - Kaitai Struct: declarative language to generate binary data parsers in C++ / C# / Go / Java / JavaScript / Lua / Nim / Perl / PHP / Python / Ruby
angle-grinder - Slice and dice logs on the command line
kaitai_struct_visualizer - Kaitai Struct: visualizer and hex viewer tool
csv2json - Simple tool for converting CSVs to JSON
ImHex - 🔍 A Hex Editor for Reverse Engineers, Programmers and people who value their retinas when working at 3 AM.
jq - Command-line JSON processor [Moved to: https://github.com/jqlang/jq]
unblob - Extract files from any kind of container formats
nq - Unix command line queue utility