Entity Resolution with Magniv

• record-linkage-resources

  1 105 0.0

  Resources for tackling record linkage / deduplication / data matching problems

  Entity resolution/record linkage/deduplication is an oddly specialized domain of knowledge given that it's such a common problem. I put together a page of resources a while back if anyone is interested: https://github.com/ropeladder/record-linkage-resources

• dedupe

  9 3,970 7.1 Python

  :id: A python library for accurate and scalable fuzzy matching, record deduplication and entity-resolution.

  

• InfluxDB

  www.influxdata.com

  sponsored

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• faiss

  70 27,924 9.4 C++

  A library for efficient similarity search and clustering of dense vectors.

  

  You just can't, similarity metrics (especially cosine) on 768 dim arrays are prohibitively slow.

  Any reason you couldn't just dump it in FAISS or Annoy? No need to do pairwise comparisons. https://github.com/facebookresearch/faiss/issues/95

Suggest a related project