gqlalchemy VS twitch-analytics-demo

As already mentioned, link prediction refers to the task of predicting missing links or links that are likely to occur in the future. In this tutorial, we will make use the of MAGE spell called node2vec. Also, we will use Memgraph to store data, and gqlalchemy to connect from a Python application. The dataset will be similar to the one used in this paper: Graph Embedding Techniques, Applications, and Performance: A Survey.

For any other service provider, it is possible to implement your custom importer class, here's how. Don't forget that GQLAlchemy is an open source project, so you can submit your extended functionality on our GitHub repository.

If you think there is something crucial that is missing or are even willing the try out your expertise in Python and graphs, check out our GitHub repository and feel free to contribute.

gqlalchemy – a Python driver and object graph mapper (OGM)

There is a broad number of drivers in many different programming languages available for both solutions. While Memgraph only maintains a few in-house drivers that it develops and supports (C, C++, Python, Rust), most Neo4j drivers can also be used with Memgraph. This is due to the fact that both solutions use the Bolt protocol, labeled property graph model and Cypher query language.

Memgraph natively has several methods of data import - import from files, MySQL or PostgreSQL, and data streams. Memgraph is also highly extendable, and with the help of its Python client, GQLAlchemy, you can import data from almost anywhere.

Source code for GQLAlchemy is available at GitHub repo.

I think building your graph database/structure can be quite an engineering and time-consuming challenge, as you mentioned, which I would personally avoid. I believe there are some solutions out there that may help you. There is one open source solution for the requirements and concerns you are mentioning. It checks out most of the things you need, functionality, efficiency, and custom low-level optimizations and it is not bulky as the Neo4j Java backend. In essence, we have built Memgraph an in-memory graph database written in C++. The distinctive key feature of DB is that all the data is stored in RAM for fast queries. There is some cool stuff with ML for graphs. Take a look at this blog post about node embedding and recommendation engines, it is native integration with Python and uses PyTorch. There is also the MAGE library for graph algorithms and ML, it is also open-sourced, which is great news for customization and expansions. I share your thoughts on OpenCypher, as being an issue. Memgraph has an object graph mapper (similar to ORM), called GQLAlchemy, and is in Python. There is also a learning curve, but not a different new skill as Cypher. The good thing is allowed various features for graphs manipulation via Python. There are also some other solutions such TigerGraph, Nebula, etc. But I am not very familiar with them. Feel free to explore. I hope this helps! 😁

Using gqlalchemy we are trying to connect to Memgraph, just like we have done before in our backend.

As expected, Flask is there, so it will be installed in our container. Next, we have pymgclient, Memgraph database adapter for Python language on top of which gqlalchemy is built. We will connect to the database with gqlalchemy and it will assist us in writing and running queries on Memgraph.

When we’re building demo applications to showcase Memgraph, we always use Docker Compose. This allows us to fire up the application on any system, which is useful when showing a demo at conferences or meetups. Also, applications created with Docker Compose are much easier to deploy. One such demo is the Twitch analytics demo. In the docker-compose.yml file, we defined a couple of services:

While building the whole web application, you save the necessary data to the database and load it whenever needed. Fetching the data can be error-prone since there may be that one minor typo that will bug you. In this tutorial, you'll learn how to build a small part of the Twitch analytics app with the help of GQLAlchemy, an object graph mapper (OGM) that makes building graph-based apps much easier.

To get started, read Part 1 and Part 2. If you want to skip that and hop right on the streaming part, you can find the backend and frontend implementations from the first two parts here.

If you still haven't, you can read already published Part 1 and then continue reading this part. Otherwise, use already implemented backend. In this part, we are going to create React application and visualize general statistics and some interesting insights from Twitch dataset. All implementation that will be mentioned in this part of the blog you can find in frontend folder of the project.

Sure, here is the scraper: https://github.com/memgraph/twitch-analytics-demo/tree/main/... and let me know if you have any questions regarding the Twitch API, since you have to make your own account :)

The data was collected using Twitch API. The data needed to be rearranged so that it could fit the idea of graph databases. Here you can find the script that creates .csv files which we'll load into Memgraph. The files which we'll use are: streamers.csv, teams.csv, vips.csv, moderators.csv and chatters.csv. In streamers.csv we can find important information about languages the user speaks and games the user streams. Those two will actually be nodes in our graph database.