GEM VS cleora

Besides, they implemented a fast C++ version of the code that works for much larger graphs. If one searches for ProNE's implementation, they would (hypothetically) find the scikit-style wrapper instead of the fully-functional release. It reminds me of a situation with HOPE, when authors of one survey "implemented" it as naive SVD (https://github.com/palash1992/GEM/blob/master/gem/embedding/hope.py#L68) instead of Jacobi-Davidson generalized solver described in the paper (and literally with code released!!). In the end, I would assume that poor paper was less cited because of that repackaging effort.

Abstract: The area of graph embeddings is currently dominated by contrastive learning methods, which demand formulation of an explicit objective function and sampling of positive and negative examples. This creates a conceptual and computational overhead. Simple, classic unsupervised approaches like Multidimensional Scaling (MSD) or the Laplacian eigenmap skip the necessity of tedious objective optimization, directly exploiting data geometry. Unfortunately, their reliance on very costly operations such as matrix eigendecomposition make them unable to scale to large graphs that are common in today's digital world. In this paper we present Cleora: an algorithm which gets the best of two worlds, being both unsupervised and highly scalable. We show that high quality embeddings can be produced without the popular step- wise learning framework with example sampling. An intuitive learning objective of our algorithm is that a node should be similar to its neighbors, without explicitly pushing disconnected nodes apart. The objective is achieved by iterative weighted averaging of node neigbors' embeddings, followed by normalization across dimensions. Thanks to the averaging operation the algorithm makes rapid strides across the embedding space and usually reaches optimal embeddings in just a few iterations. Cleora runs faster than other state-of-the-art CPU algorithms and produces embeddings of competitive quality as measured on downstream tasks: link prediction and node classification. We show that Cleora learns a data abstraction that is similar to contrastive methods, yet at much lower computational cost. We open-source Cleora under the MIT license allowing commercial use under this https URL.

Our team at Synerise AI has open sourced Cleora - an ultra fast vertex embedding tool for graphs & hypergraphs. If you've ever used node2vec, DeepWalk, LINE or similar methods - it might be worth to check it out.

Thanks for raising so many interesting points about model performance and complexity. In this context, I think our newly released graph embedding library - Cleora - might be of interest: https://github.com/Synerise/cleora Cleora has some nice performance-wise properties:

Synerise/cleora (Rust): Cleora AI is a general-purpose model for efficient, scalable learning of stable and inductive entity embeddings for heterogeneous relational data.