hckrweb VS splink

I tried sharing a couple of my web apps:

- HN the way I want to read it: https://hw.leftium.com/

- Source code: https://github.com/Leftium/hckrweb

- Weather forecast compared to last two days' weather: https://github.com/Leftium/ultra-weather#readme

HckrWeb is my heavily modified fork of HackerWeb[1] with data from hckr news[2].

I mashed together the simple, readable UI of HackerWeb with the chronological ordering from hckr news.

You can also share links to readable HN comments like this: https://hw.leftium.com/#/item/23741256

Source code available here: https://github.com/Leftium/hckrweb

[1]: https://hackerwebapp.com/

https://github.com/moj-analytical-services/splink

Record linkage has been a big part of a project I've been working on for 6 months now. I personally think a great and free solution be using the splink package in Python which can handle 10+m rows which implements the Fellegi-Sunter model (equivalent to a naive-Bayes model) is the classical model in record linkage. It can be trained in an unsupervised manner using some initial parameter estimation (these are quite intuitive) and then expectation maximisation. The features in the model will be different pairwise string comparisons on your field of interest. These can include exact equality; edit distance comparisons like Levensthein distance and Jaro-Winkler; and phonetic comparisons like soundex and double metaphone. The splink pacakge will handle training the model and then all the graph theory at the end to connect all your links into clusters. All the details you'll need are in the links. https://www.robinlinacre.com/probabilistic\_linkage/ https://moj-analytical-services.github.io/splink/

https://moj-analytical-services.github.io/splink/ is a FOSS python package (but it runs against your db using SQL).

If you're curious, I've written a FOSS record linkage library that executes everything as SQL. It supports multiple SQL backends including DuckDB and Spark for scale, and runs faster than most competitors because it's able to leverage the speed of these backends: https://github.com/moj-analytical-services/splink

Splink - a python library for probabilistic record linkage (fuzzy matching/entity resolution).

Splink is dramatically faster and works on much larger datasets than other open source libraries. I'm particularly proud of the fact we support multiple execution backends (at the moment, DuckDb Spark Athena and Sqlite, but additional adaptors are relatively straightforward to write).

We've had >4 million pypi downloads and it's used in government, academia and the private sector, often replacing extremely expensive proprietary solutions.

https://github.com/moj-analytical-services/splink

More info in blog posts here:

Splink is a SQL tool that can do this https://github.com/moj-analytical-services/splink

Probabilistic record matching model such as a Fellegi-Sunter. Check out the splink package in Python.

Main docs here: https://moj-analytical-services.github.io/splink

It'd be great to see Splink add value in this area! Do give us a shout if you have any questions. The best place to post is on the Github discussions: https://github.com/moj-analytical-services/splink/discussions