pyWhat
hachoir
Our great sponsors
| pyWhat | hachoir | |
|---|---|---|
| 16 | 3 | |
| 6,346 | 588 | |
| - | - | |
| 0.0 | 6.4 | |
| 6 months ago | 2 months ago | |
| Python | Python | |
| MIT License | GNU General Public License v3.0 only |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
pyWhat
-
Go Library like PyWhat?
Is there a library written in Go similar to PyWhat? I want to use a subset of the functionality for a simple go program I'm writing. I could just call PyWhat, link to lemmeknow, or even write a simple go implementation myself, but I wanted to ask if there was a pure go implementation. Thanks!
-
lemmeknow v0.7.0 is here with support for identifying bytes with help of regex crate!
Lemmeknow is basically used for identifying text as mentioned in README and video. It is Rust implementation of PyWhat. You can see various usecases there too.
-
lemmeknow - The fastest way to identify anything!
For rarity, we have got the database from pyWhat and the wiki says:
This project is inspired by PyWhat! Thanks to developer of it for the awesome idea <3 . Lemmeknow is blazingly faster than PyWhat btw ;)
-
lemmeknow - the fastest way to identify anything!
This project was inspired by u/beesec 's pyWhat
- Tips for Making a Popular Open-Source Project in 2021 [Ultimate Guide]
-
IT Pro Tuesday #155 - Carrier Lookup, Network Podcast, Identification Tool & More
pyWhat enables you to easily identify emails, IP addresses and more. Feed it a .pcap file or some mysterious text or hex of a file, and it will tell you what it is. The tool is recursive, so it can identify everything in text, files and more. A shout out to the tool's author for sharing his creation.
- Identify anything. pyWhat easily lets you identify emails, IP addresses, and more. Feed it a .pcap file or some text and it'll tell you what it is
- PyWhat: Identify Anything
hachoir
- Magika: AI powered fast and efficient file type identification
-
Kaitai Struct: A new way to develop parsers for binary structures
I contributed a number of file formats a few years ago (and attempted numerous others) but ran into a number of problems with certain file formats:
1. It's not possible to read from the file until a multiple byte termination sequence is detected. [1]
2. You can't read sections of a file where the termination condition is the presence of a sequence of bytes denoting the next unrelated section of the file (and you don't want to consume/read these bytes) [2]
3. The WebIDE at the time couldn't handle very large file format specifications such as Photoshop (PSD) [3]
4. Files containing compressed or encrypted sections require a compression/encryption algorithm to be hardcoded into Kaitai struct libraries for each programming language it can output to.
The WebIDE I particularly liked as it makes it easy to get started and share results. I also liked how Kaitai Struct allows easy definition of constraints (simple ones at least) into the file format specification so that you can say "this section of the file shall have a size not exceeding header.length * 2 bytes".
Some alternative binary file format specification attempts for those interested in seeing alternatives, each with their own set of problems/pros/cons:
1. 010 Editor [4]
2. Synalysis [5]
3. hachoir [6]
4. DFDL [7]
[1] https://github.com/kaitai-io/kaitai_struct/issues/158
[2] https://github.com/kaitai-io/kaitai_struct/issues/156
[3] https://raw.githubusercontent.com/davidhicks/kaitai_struct_f...
[4] https://www.sweetscape.com/010editor/repository/templates/
[5] https://github.com/synalysis/Grammars
[6] https://github.com/vstinner/hachoir/tree/main/hachoir/parser
-
PyWhat: Identify Anything
Another one sort of related is hachoir, and specifically the hachoir-metadata script: https://github.com/vstinner/hachoir
What are some alternatives?
arkime - Arkime is an open source, large scale, full packet capturing, indexing, and database system.
BruteShark - Network Analysis Tool
chepy - Chepy is a python lib/cli equivalent of the awesome CyberChef tool.
TryHackMe - This is a repository containing TryHackMe Writeups in Somali language on various of rooms & challenges, including notes, files and solutions.
usaddress - :us: a python library for parsing unstructured United States address strings into address components
maltrail - Malicious traffic detection system
dumpulator - An easy-to-use library for emulating memory dumps. Useful for malware analysis (config extraction, unpacking) and dynamic analysis in general (sandboxing).
DataProfiler - What's in your data? Extract schema, statistics and entities from datasets
Ciphey - ⚡ Automatically decrypt encryptions without knowing the key or cipher, decode encodings, and crack hashes ⚡
ViperMonkey - A VBA parser and emulation engine to analyze malicious macros.
binrw - A Rust crate for helping parse and rebuild binary data using ✨macro magic✨.
chardet - Python character encoding detector