sic1 VS splink

Whoa looks nice. Assembly-like programming games (like TIS-100) is always interesting for me.

https://github.com/jaredkrinke/sic1/tree/master/sic1

My single-instruction (subleq) programming game:

https://github.com/jaredkrinke/sic1

I really thought enough people liked esolangs and zachlikes, but it failed to get a single upvote, so never even made it to the “Show” page (well, not until like a week later, at which point it was buried anyway) :(

https://github.com/moj-analytical-services/splink

Record linkage has been a big part of a project I've been working on for 6 months now. I personally think a great and free solution be using the splink package in Python which can handle 10+m rows which implements the Fellegi-Sunter model (equivalent to a naive-Bayes model) is the classical model in record linkage. It can be trained in an unsupervised manner using some initial parameter estimation (these are quite intuitive) and then expectation maximisation. The features in the model will be different pairwise string comparisons on your field of interest. These can include exact equality; edit distance comparisons like Levensthein distance and Jaro-Winkler; and phonetic comparisons like soundex and double metaphone. The splink pacakge will handle training the model and then all the graph theory at the end to connect all your links into clusters. All the details you'll need are in the links. https://www.robinlinacre.com/probabilistic\_linkage/ https://moj-analytical-services.github.io/splink/

https://moj-analytical-services.github.io/splink/ is a FOSS python package (but it runs against your db using SQL).

If you're curious, I've written a FOSS record linkage library that executes everything as SQL. It supports multiple SQL backends including DuckDB and Spark for scale, and runs faster than most competitors because it's able to leverage the speed of these backends: https://github.com/moj-analytical-services/splink

Splink - a python library for probabilistic record linkage (fuzzy matching/entity resolution).

Splink is dramatically faster and works on much larger datasets than other open source libraries. I'm particularly proud of the fact we support multiple execution backends (at the moment, DuckDb Spark Athena and Sqlite, but additional adaptors are relatively straightforward to write).

We've had >4 million pypi downloads and it's used in government, academia and the private sector, often replacing extremely expensive proprietary solutions.

https://github.com/moj-analytical-services/splink

More info in blog posts here:

Splink is a SQL tool that can do this https://github.com/moj-analytical-services/splink

Probabilistic record matching model such as a Fellegi-Sunter. Check out the splink package in Python.

Main docs here: https://moj-analytical-services.github.io/splink

It'd be great to see Splink add value in this area! Do give us a shout if you have any questions. The best place to post is on the Github discussions: https://github.com/moj-analytical-services/splink/discussions