cugraph VS gqlalchemy

cuGraph does some of what Networkx does, but it is far from being as easy to use. But it should be fast.

https://en.wikipedia.org/wiki/Sparse_matrix :

> The concept of sparsity is useful in combinatorics and application areas such as network theory and numerical analysis, which typically have a low density of significant data or connections. Large sparse matrices often appear in scientific or engineering applications when solving partial differential equations.

CuGraph has a NetworkX-like API, though only so many of the networkx algorithms are CUDA-optimized.

https://github.com/rapidsai/cugraph :

> cuGraph operates, at the Python layer, on GPU DataFrames, thereby allowing for seamless passing of data between ETL tasks in cuDF and machine learning tasks in cuML. Data scientists familiar with Python will quickly pick up how cuGraph integrates with the Pandas-like API of cuDF. Likewise, users familiar with NetworkX will quickly recognize the NetworkX-like API provided in cuGraph, with the goal to allow existing code to be ported with minimal effort into RAPIDS.

> While the high-level cugraph python API provides an easy-to-use and familiar interface for data scientists that's consistent with other RAPIDS libraries in their workflow, some use cases require access to lower-level graph theory concepts. For these users, we provide an additional Python API called pylibcugraph, intended for applications that require a tighter integration with cuGraph at the Python layer with fewer dependencies. Users familiar with C/C++/CUDA and graph structures can access libcugraph and libcugraph_c for low level integration outside of python.

/? sparse

I feel like you would need to develop a custom solution which might in part store data in Neo4j but you will have to figure out how to efficiently pull the data you need to train your GNNs; and I think this tends to be the bottleneck since Graph DBs are not optimised for the kinds of queries you need for GNNs. For what it's worth, I wouldn't really bother with implementing a custom graph data structure (unless I was really keen) as there are some good implementations out there. Have you looked at cuGraph for example?

As already mentioned, link prediction refers to the task of predicting missing links or links that are likely to occur in the future. In this tutorial, we will make use the of MAGE spell called node2vec. Also, we will use Memgraph to store data, and gqlalchemy to connect from a Python application. The dataset will be similar to the one used in this paper: Graph Embedding Techniques, Applications, and Performance: A Survey.

For any other service provider, it is possible to implement your custom importer class, here's how. Don't forget that GQLAlchemy is an open source project, so you can submit your extended functionality on our GitHub repository.

If you think there is something crucial that is missing or are even willing the try out your expertise in Python and graphs, check out our GitHub repository and feel free to contribute.

gqlalchemy – a Python driver and object graph mapper (OGM)

There is a broad number of drivers in many different programming languages available for both solutions. While Memgraph only maintains a few in-house drivers that it develops and supports (C, C++, Python, Rust), most Neo4j drivers can also be used with Memgraph. This is due to the fact that both solutions use the Bolt protocol, labeled property graph model and Cypher query language.

Memgraph natively has several methods of data import - import from files, MySQL or PostgreSQL, and data streams. Memgraph is also highly extendable, and with the help of its Python client, GQLAlchemy, you can import data from almost anywhere.

Source code for GQLAlchemy is available at GitHub repo.

I think building your graph database/structure can be quite an engineering and time-consuming challenge, as you mentioned, which I would personally avoid. I believe there are some solutions out there that may help you. There is one open source solution for the requirements and concerns you are mentioning. It checks out most of the things you need, functionality, efficiency, and custom low-level optimizations and it is not bulky as the Neo4j Java backend. In essence, we have built Memgraph an in-memory graph database written in C++. The distinctive key feature of DB is that all the data is stored in RAM for fast queries. There is some cool stuff with ML for graphs. Take a look at this blog post about node embedding and recommendation engines, it is native integration with Python and uses PyTorch. There is also the MAGE library for graph algorithms and ML, it is also open-sourced, which is great news for customization and expansions. I share your thoughts on OpenCypher, as being an issue. Memgraph has an object graph mapper (similar to ORM), called GQLAlchemy, and is in Python. There is also a learning curve, but not a different new skill as Cypher. The good thing is allowed various features for graphs manipulation via Python. There are also some other solutions such TigerGraph, Nebula, etc. But I am not very familiar with them. Feel free to explore. I hope this helps! 😁

Using gqlalchemy we are trying to connect to Memgraph, just like we have done before in our backend.

As expected, Flask is there, so it will be installed in our container. Next, we have pymgclient, Memgraph database adapter for Python language on top of which gqlalchemy is built. We will connect to the database with gqlalchemy and it will assist us in writing and running queries on Memgraph.