Similar projects and alternatives to zingg based on common topics and language
Less time debugging, more time building. Scout APM allows you to find and fix performance issues with no hassle. Now with error monitoring and external services monitoring, Scout is a developer's best friend when it comes to application development.
zingg reviews and mentions
is it possible to "fuzzy match" or dedupe columns in Redshift?
1 project | reddit.com/r/aws | 30 Apr 2022
If you are open to using a framework for this, check Zingg at https://github.com/zinggAI/zingg. It connects to Redshift, snowflake and other warehouses and can handle multiple columns
Show HN: Zingg – open-source entity resolution for single source of truth
3 projects | news.ycombinator.com | 9 Feb 2022
I am Sonal, a data consultant from India. For the past few months(and years!), I have been working on an entity resolution tool to build a single source of truth for customers, suppliers, products and parts. Here is a short demo of Zingg in action https://www.youtube.com/watch?v=zOabyZxN9b0
As a data consultant, I often struggled to build unified views of core entities on the datalake and the warehouse. Data spread across different systems has variations and consistencies making Customer 360, KYC, AML, segmentation, personalization and other analytics difficult.
As I talked with different clients facing this issue, I searched for existing solutions which I could use or recommend. Unfortunately, most of them were very expensive MDM solutions like Tamr, or CDP solutions like Amperity. There were many open source libraries, but they did not tie well into the datalake/warehouse scenarios we were working with, did not scale and/or needed a decent bit of programming or did not generalize. I even tried to build something internally and failed miserably, and that got me hooked :-)
As I dug deeper into the problem, I realized that there were multiple challenges. Data matching, at its very core, becomes a cartesian join, as you need to compare every pair of records to figure out the matches. With millions of records, this becomes extremely tough to scale. I referred to various research papers and then implemented a blocking algorithm to overcome this. More details at https://docs.zingg.ai/docs/zModels.html
The second challenge was to say which pairs are a match. I wanted to have a machine learning-based approach to handle the different types of entities and the variety of differences in real world data. But I also felt that non ML experts should be able to use Zingg easily, hence took the approach of abstracting the feature generation and hyper-parameter tuning for the classifier.
Once I settled on the ML approach, the problem of training data quickly arose, which led me to pick up active learning and build an interactive labeler through which sample records can be marked as matches and non matches to build training sets quickly. I still feel that we should have an unsupervised approach as well, but I have not yet figured out the right way to do so.
The Zingg repository is hosted at https://github.com/zinggAI/zingg and we have close to 60 members on our Slack(https://join.slack.com/t/zinggai/shared_invite/zt-w7zlcnol-vEuqU9m~Q56kLLUVxRgpOA). We are now two developers working full time on Zingg!!! I am super happy that early users have been able to use Zingg and push us to build more stuff - model documentation, using pre-existing training data, native Snowflake integration etc.
I have been an open source consumer all my dev life, and this is the first time I have made a decent contribution. It is my first time trying to build a community as well. Not sure how the future will unfold, but wanted to reach out to the community here and hear what you think about the problem, the approach, any ideas or suggestions.
Thanks for reading along, and please do post your thoughts in the comments below.3 projects | news.ycombinator.com | 9 Feb 2022
Thanks for your support. Yes we do ship with some examples and their models which can be run out of the box. We have 3 customer demographic datasets and an ecommerce items matching across Google and Amazon. You can check them here https://github.com/zinggAI/zingg/tree/main/examples
Question about Github Referring Sites
1 project | reddit.com/r/github | 30 Dec 2021
I have an open source project hosted at https://github.com/zinggAI/zingg/.
How do I promote the project appropriately?
3 projects | reddit.com/r/opensource | 30 Dec 2021
Have you tried posting on hacker news and subreddits? I am also working on an open source entity resolution tool at https://github.com/zinggAI/zingg and I saw good response from the data engineering, data science and the ML subreddits as well as hacker news.3 projects | reddit.com/r/opensource | 30 Dec 2021
GitHub Java Projects to Contribute
2 projects | reddit.com/r/opensource | 17 Nov 2021
Check Zingg out at https://github.com/zinggAI/zingg and let me know if you would like to contribute
Match over 1 GB of data with inconsistent names
3 projects | reddit.com/r/dataengineering | 9 Nov 2021
I am working on an open source tool that uses ml for fuzzy matching - https://github.com/zinggAI/zingg . Hope you find it useful. Happy to help.3 projects | reddit.com/r/dataengineering | 9 Nov 2021
This is interesting, would love to get your feedback on Zingg(https://github.com/zinggAI/zingg) if you are upto it. Thanks!
Open source entity resolution - need your feedback!
1 project | reddit.com/r/bigdata | 8 Nov 2021
I have released an open source entity resolution tool Zingg(https://github.com/zinggAI/zingg). Zingg uses Spark and ML to build single source of truth directly in the warehouse or the datalake. Would love to hear from the Reddit folks here what they think about it - do you find it useful? what can I do to make it better? any advice on the problem or the solution?
Data Curation, master data tools
1 project | reddit.com/r/opensource | 26 Oct 2021
I am working on Zingg(https://github.com/zinggAI/zingg ) for matching records. Would love to extend it and help you by building the dq functions if you need
What are some cool data science projects you have done or come across?
1 project | reddit.com/r/datascience | 16 Oct 2021
I am working on an open source entity resolution tool https://github.com/zinggAI/zingg
Introducing Zingg: Open Source Entity Resolution and Deduplication Using ML and Spark
2 projects | reddit.com/r/datascience | 5 Oct 2021
- Zingg scales very well to large volumes of data(https://github.com/zinggAI/zingg/blob/main/docs/hardwareSizing.md)2 projects | reddit.com/r/datascience | 5 Oct 2021
1 project | reddit.com/r/snowflake | 30 Sep 2021
zinggAI/zingg is an open source project licensed under GNU Affero General Public License v3.0 which is an OSI approved license.