jellyfish
ceja
Our great sponsors
jellyfish | ceja | |
---|---|---|
3 | 1 | |
1,990 | 32 | |
- | - | |
6.8 | 2.3 | |
about 1 month ago | 2 months ago | |
Jupyter Notebook | Python | |
MIT License | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
jellyfish
-
Python Libraries
For sounds something like https://github.com/jamesturk/jellyfish ?
-
Comparing Strings (Street Names) With Machine Learning
When comparing strings (in our case street names), there are plenty of off-the-shelf features that can be used, such as those provided by the jellyfish. This package also provides a number of phonetic encodings. We can combine an encoding with a metric, such as Levenshtein Distance, to measure the phonetic similarity between two street names.
-
How to match names which differ slightly?
You can use a library like jellyfish which implements a bunch of string comparison algorithms, you'd just have to experiment and see which one gives the best results for you. I think I've had the best luck with Jaro-Winkler, then looking at the % match result and picking a cutoff above which I have good confidence that the match is real. It's still not perfect, and I really don't see how your last example would work with just about any automated comparison.
ceja
-
Which language is better for Spark & Why?
Some Python libraries are easily wrapped via UDFs, see ceja, but don't overestimate this benefit. Most Python libraries cannot be wrapped and run via Spark in a performant manner.
What are some alternatives?
fuzzywuzzy - Fuzzy String Matching in Python
TextDistance - 📐 Compute distance between sequences. 30+ algorithms, pure python implementation, common interface, optional external libs usage.
Apache Spark - Apache Spark - A unified analytics engine for large-scale data processing
Levenshtein - The Levenshtein Python C extension module contains functions for fast computation of Levenshtein distance and string similarity
textdistance.rs - 🦀📏 Rust library to compare strings (or any sequences). 25+ algorithms, pure Rust, common interface, Unicode support.
Pygments
sparkmagic - Jupyter magics and kernels for working with remote Spark clusters
汉字拼音转换工具(Python 版) - 汉字转拼音(pypinyin)
pydantic - Data validation using Python type hints
pyfiglet - An implementation of figlet written in Python
python-slugify - Returns unicode slugs