hachoir
usaddress
Our great sponsors
| hachoir | usaddress | |
|---|---|---|
| 3 | 5 | |
| 587 | 1,485 | |
| - | 0.7% | |
| 6.4 | 0.0 | |
| 2 months ago | 3 months ago | |
| Python | Python | |
| GNU General Public License v3.0 only | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
hachoir
- Magika: AI powered fast and efficient file type identification
-
Kaitai Struct: A new way to develop parsers for binary structures
I contributed a number of file formats a few years ago (and attempted numerous others) but ran into a number of problems with certain file formats:
1. It's not possible to read from the file until a multiple byte termination sequence is detected. [1]
2. You can't read sections of a file where the termination condition is the presence of a sequence of bytes denoting the next unrelated section of the file (and you don't want to consume/read these bytes) [2]
3. The WebIDE at the time couldn't handle very large file format specifications such as Photoshop (PSD) [3]
4. Files containing compressed or encrypted sections require a compression/encryption algorithm to be hardcoded into Kaitai struct libraries for each programming language it can output to.
The WebIDE I particularly liked as it makes it easy to get started and share results. I also liked how Kaitai Struct allows easy definition of constraints (simple ones at least) into the file format specification so that you can say "this section of the file shall have a size not exceeding header.length * 2 bytes".
Some alternative binary file format specification attempts for those interested in seeing alternatives, each with their own set of problems/pros/cons:
1. 010 Editor [4]
2. Synalysis [5]
3. hachoir [6]
4. DFDL [7]
[1] https://github.com/kaitai-io/kaitai_struct/issues/158
[2] https://github.com/kaitai-io/kaitai_struct/issues/156
[3] https://raw.githubusercontent.com/davidhicks/kaitai_struct_f...
[4] https://www.sweetscape.com/010editor/repository/templates/
[5] https://github.com/synalysis/Grammars
[6] https://github.com/vstinner/hachoir/tree/main/hachoir/parser
-
PyWhat: Identify Anything
Another one sort of related is hachoir, and specifically the hachoir-metadata script: https://github.com/vstinner/hachoir
usaddress
-
Which of your favorite Python 3.11 packages lack Python 3.11 support?
Usaddress https://github.com/datamade/usaddress
-
Script to split addresses in Google Sheets?
Assuming you’re working with addresses in the US, here’s a Python package that should help: https://github.com/datamade/usaddress
-
PyWhat: Identify Anything
Some great probabilistic python libraries:
https://github.com/datamade/usaddress - "usaddress is a Python library for parsing unstructured address strings into address components, using advanced NLP methods."
https://github.com/datamade/probablepeople - "probablepeople is a python library for parsing unstructured romanized name or company strings into components, using advanced NLP methods."
- Turning unstructured address data into a structure Salesforce Address Field
-
Fuzzy Name Matching in Postgres
For address parsing, I've had good luck with this package: https://github.com/datamade/usaddress
What are some alternatives?
binrw - A Rust crate for helping parse and rebuild binary data using ✨macro magic✨.
libpostal - A C library for parsing/normalizing street addresses around the world. Powered by statistical NLP and open geo data.
fuckitjs - The Original Javascript Error Steamroller
pyWhat - 🐸 Identify anything. pyWhat easily lets you identify emails, IP addresses, and more. Feed it a .pcap file or some text and it'll tell you what it is! 🧙♀️
probablepeople - :family: a python library for parsing unstructured western names into name components.
DataProfiler - What's in your data? Extract schema, statistics and entities from datasets
smm2-documentation - Documentation for the game Super Mario Maker 2.
ctparse - Parse natural language time expressions in python
chardet - Python character encoding detector
SymSpell - SymSpell: 1 million times faster spelling correction & fuzzy search through Symmetric Delete spelling correction algorithm