hachoir
file
hachoir | file | |
---|---|---|
3 | 14 | |
586 | 1,178 | |
- | 1.1% | |
6.4 | 9.2 | |
3 months ago | 21 days ago | |
Python | C | |
GNU General Public License v3.0 only | GNU General Public License v3.0 or later |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
hachoir
-
Magika: AI powered fast and efficient file type identification
https://github.com/vstinner/hachoir/blob/main/hachoir/subfil...
File signature:
-
Kaitai Struct: A new way to develop parsers for binary structures
I contributed a number of file formats a few years ago (and attempted numerous others) but ran into a number of problems with certain file formats:
1. It's not possible to read from the file until a multiple byte termination sequence is detected. [1]
2. You can't read sections of a file where the termination condition is the presence of a sequence of bytes denoting the next unrelated section of the file (and you don't want to consume/read these bytes) [2]
3. The WebIDE at the time couldn't handle very large file format specifications such as Photoshop (PSD) [3]
4. Files containing compressed or encrypted sections require a compression/encryption algorithm to be hardcoded into Kaitai struct libraries for each programming language it can output to.
The WebIDE I particularly liked as it makes it easy to get started and share results. I also liked how Kaitai Struct allows easy definition of constraints (simple ones at least) into the file format specification so that you can say "this section of the file shall have a size not exceeding header.length * 2 bytes".
Some alternative binary file format specification attempts for those interested in seeing alternatives, each with their own set of problems/pros/cons:
1. 010 Editor [4]
2. Synalysis [5]
3. hachoir [6]
4. DFDL [7]
[1] https://github.com/kaitai-io/kaitai_struct/issues/158
[2] https://github.com/kaitai-io/kaitai_struct/issues/156
[3] https://raw.githubusercontent.com/davidhicks/kaitai_struct_f...
[4] https://www.sweetscape.com/010editor/repository/templates/
[5] https://github.com/synalysis/Grammars
[6] https://github.com/vstinner/hachoir/tree/main/hachoir/parser
[7] https://github.com/DFDLSchemas/
-
PyWhat: Identify Anything
Another one sort of related is hachoir, and specifically the hachoir-metadata script: https://github.com/vstinner/hachoir
file
- Magika: AI powered fast and efficient file type identification
-
Who invented file extensions in file names?
https://www.darwinsys.com/file/
This is the authors website. Apparently yeah its not part of GNU utils, I had no idea, I knew it came with most Linux systems so I looked for the Debian package and found the site linked above.
https://packages.debian.org/bookworm/file
-
TIL: Github seems to recognize ebuilds as a format. Is this a new github feature? Or has this been here since forever?
https://github.com/file/file/blob/master/magic/Magdir/gentoo https://www.iana.org/assignments/media-types/application/vnd.gentoo.ebuild (among others)
-
ELI5: Why do .jpg and .jpeg both exist?
Look into libmagic.
-
How do I make a transparent background on .ps or .eps file imported to groff
in installations of Linux, file is a command to retrieve some basic information about a file type. Because contrasting to the organization of Windows, the file extension in Linux can be less important for the file's use (even irrelevant), than the actual content.
-
Why does Linux file command shows java file as C++ source?
In this case, it's hitting this set of rules. I would guess it hits both class and public.
- Linux `file` Equivalent
-
How many “encodings” are there?
But I don't think this is what you meant. I think your question is something more like: how many file formats are there? The Unix file command recognizes several thousand, and Wikipedia has a list. Of course these are only the most common, and it's hard to delineate what counts as a unique file format. (Is YAML one file format or millions?)
-
Can I change the mime type of OGG Vorbis and Opus files without re-encoding them?
The magic number for all ogg files is the same (OggS) and files magic file looks like it decides the mime type based on the codecs used in the file. However, it looks like application/ogg is used only if the file contains Kate (Karaoke and Text) content.
-
Fun with File Formats
Also the magic number database for guessing the format of a file:
https://www.darwinsys.com/file/
What are some alternatives?
binrw - A Rust crate for helping parse and rebuild binary data using ✨macro magic✨.
magika - Detect file content types with deep learning
usaddress - :us: a python library for parsing unstructured United States address strings into address components
SheetJS js-xlsx - 📗 SheetJS Spreadsheet Data Toolkit -- New home https://git.sheetjs.com/SheetJS/sheetjs
fuckitjs - The Original Javascript Error Steamroller
file-windows - File and Libmagic build with Visual Studio
pyWhat - 🐸 Identify anything. pyWhat easily lets you identify emails, IP addresses, and more. Feed it a .pcap file or some text and it'll tell you what it is! 🧙♀️
tika-docker - Convenience Docker images for Apache Tika Server
probablepeople - :family: a python library for parsing unstructured western names into name components.
tablib - Python Module for Tabular Datasets in XLS, CSV, JSON, YAML, &c.
smm2-documentation - Documentation for the game Super Mario Maker 2.
rust_dos - Rust DOS : Creating a DOS executable with Rust