file
tablib
Our great sponsors
file | tablib | |
---|---|---|
14 | 2 | |
1,154 | 4,509 | |
3.2% | 1.3% | |
9.2 | 7.0 | |
19 days ago | 6 days ago | |
C | Python | |
GNU General Public License v3.0 or later | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
file
-
Magika: AI powered fast and efficient file type identification
As someone that has worked in a space that has to deal with uploaded files for the last few years, and someone who maintains a WASM libmagic Node package ( https://github.com/moshen/wasmagic ) , I have to say I really love seeing new entries into the file type detection space.
Though I have to say when looking at the Node module, I don't understand why they released it.
Their docs say it's slow:
https://github.com/google/magika/blob/120205323e260dad4e5877...
It loads the model an runtime:
https://github.com/google/magika/blob/120205323e260dad4e5877...
They mark it as Experimental in the documentation, but it seems like it was just made for the web demo.
Also as others have mentioned. The model appears to only detect 116 file types:
https://github.com/google/magika/blob/120205323e260dad4e5877...
Where libmagic detects... a lot. Over 1600 last time I checked:
https://github.com/file/file/tree/4cbd5c8f0851201d203755b76c...
I guess I'm confused by this release. Sure it detected most of my list of sample files, but in a sample set of 4 zip files, it misidentified one.
This is false in every sense for https://www.darwinsys.com/file/ (probably the most used file version). It depends on the magic for a specific file, but it can check any part of your file. Many Linux distros are years out of date, you might be using a very old version.
FILE_45:
./src/file -m magic/magic.mgc ../../OpenCalc.v2.3.1.apk
-
Who invented file extensions in file names?
https://www.darwinsys.com/file/
This is the authors website. Apparently yeah its not part of GNU utils, I had no idea, I knew it came with most Linux systems so I looked for the Debian package and found the site linked above.
-
TIL: Github seems to recognize ebuilds as a format. Is this a new github feature? Or has this been here since forever?
https://github.com/file/file/blob/master/magic/Magdir/gentoo https://www.iana.org/assignments/media-types/application/vnd.gentoo.ebuild (among others)
- Linux `file` Equivalent
-
Fun with File Formats
Also the magic number database for guessing the format of a file:
tablib
-
Fun with File Formats
There are two problems leading to the decision of only accepting public domain info: licensing and provenance.
"Licensing" is hard. The "Open Specifications Promise" [1], which covers a bunch of Microsoft-designed file formats, is merely a covenant not to sue.
"Provenance" is tricky. For example, much of the knowledge of the Apple iWork formats were derived by reverse-engineering the source programs and extracting protobuf definitions. Many open source projects have freely copied from each other, making detailed analysis tricky [2].
[1] https://en.wikipedia.org/wiki/Microsoft_Open_Specification_P...
What are some alternatives?
pymorphy2 - Morphological analyzer / inflection engine for Russian and Ukrainian languages.
Kaitai Struct - Kaitai Struct: declarative language to generate binary data parsers in C++ / C# / Go / Java / JavaScript / Lua / Nim / Perl / PHP / Python / Ruby
magika - Detect file content types with deep learning
tika-docker - Convenience Docker images for Apache Tika Server
feather - Feather: fast, interoperable binary data frame storage for Python, R, and more powered by Apache Arrow
SheetJS js-xlsx - ๐ SheetJS Spreadsheet Data Toolkit -- New home https://git.sheetjs.com/SheetJS/sheetjs
file-windows - File and Libmagic build with Visual Studio
linuxontheweb
DistorteD - Ruby multimedia toolkit with deep Jekyll integration ๐งช
fuzzywuzzy - Fuzzy String Matching in Python
linguist - Language Savant. If your repository's language is being reported incorrectly, send us a pull request!