ldbc_snb_datagen_spark
Docker Compose
ldbc_snb_datagen_spark | Docker Compose | |
---|---|---|
5 | 390 | |
165 | 32,471 | |
1.8% | 1.0% | |
3.7 | 9.6 | |
15 days ago | 8 days ago | |
Java | Go | |
Apache License 2.0 | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
ldbc_snb_datagen_spark
-
Benchgraph Backstory: The Untapped Potential
Because of the size, complexity, and feedback from the community, we decided to add a larger dataset. So the next dataset should be large, more complex, and recognizable. The choice was easy here; the industry-leading benchmark group Linked Data Benchmark Council (LDBC), which Memgraph is a part of, has open-sourced the datasets for benchmarking. The exact dataset is the social network dataset. It is a synthetically generated dataset representing a social network. It is being used in LDBC audited benchmarks, SNB interactive, and SNB Buissines intelligence benchmarks. Keep in mind that this is NOT an official implementation of an LDBC benchmark, the open-source dataset is being used as a basis for benchmarks, and it will be used for our in-house testing process and improving Memgraph.
-
Postgres: The Graph Database You Didn't Know You Had
I designed and maintain several graph benchmarks in the Linked Data Benchmark Council, including workloads aimed for databases [1]. We make no restrictions on implementations, they can any query language like Cypher, SQL, etc.
In our last benchmark aimed at analytical systems [2], we found that SQL queries using WITH RECURSIVE can work for expressing reachability and even weighted shortest path queries. However, formulating an efficient algorithm yields very complex SQL queries [3] and their execution requires a system with a sophisticated optimizer such as Umbra developed at TU Munich [4]. Industry SQL systems are not yet at this level but they may attain that sometime in the future.
Another direction to include graph queries in SQL is the upcoming SQL/PGQ (Property Graph Queries) extension. I'm involved in a project at CWI Amsterdam to incorporate this language into DuckDB [5].
[1] https://ldbcouncil.org/benchmarks/snb/
[2] https://www.vldb.org/pvldb/vol16/p877-szarnyas.pdf
[3] https://github.com/ldbc/ldbc_snb_bi/blob/main/umbra/queries/...
[4] https://umbra-db.com/
[5] https://www.cidrdb.org/cidr2023/slides/p66-wolde-slides.pdf
- Bullshit Graph Database Performance Benchmarks
-
From Data Preprocessing to Using Graph Database
Pull the source code from https://github.com/ldbc/ldbc_snb_datagen/tree/stable.To generate data for scale factor 1-1000, use the stable branch.
Docker Compose
-
Deploy a Grafana dashboard with Docker on AWS EC2
sudo curl -L https://github.com/docker/compose/releases/latest/download/docker-compose-$(uname -s)-$(uname -m) -o /usr/local/bin/docker-compose sudo chmod +x /usr/local/bin/docker-compose
- Docker Compose: `version` is obsolete
-
12 Factor: 13 years later
Solutions are many, and could include Docker Compose, VS Code dev containers, Telepresence, Localstack or setting up temporary AWS accounts as a development environment for serverless applications.
-
Let's write a simple microservice in Clojure
Using Docker Compose to run Postgres and any third-party services locally provides a streamlined and consistent development environment. Developers can define services in a docker-compose.yml file, which enables them to configure and launch an entire stack with a single command. In this case, Postgres is encapsulated within a container with predefined configurations. Docker Compose also facilitates easy scaling, updates, and isolation of services, enhancing development efficiency and reducing the setup time for new team members or transitioning between projects. It encapsulates complex configurations, such as Postgres' performance monitoring and logging settings, in a manageable, version-controlled file, simplifying and replicating the service setup across different environments.
- Live reload em Go com docker e compile daemon
-
Docker compose, orchestrating and automating services
“Compose simplifies the control of your entire application stack, making it easy to manage services, networks, and volumes in a single, comprehensible YAML configuration file. Then, with a single command, you create and start all the services from your configuration file.” - Docker documentation
-
Hosting a simple docker-compose app with Nginx and generate a SSL with certbot on digitalocean droplet
curl -fsSL https://get.docker.com -o get-docker.sh sudo sh get-docker.sh # Install docker compose sudo curl -L "https://github.com/docker/compose/releases/download/1.29.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose # Apply executable permissions to the binary sudo chmod +x /usr/local/bin/docker-compose # Run Project docker-compose up -d
-
One Minute: Compose
Docker,
-
How to Set Up a Docker Container
This foundation now opens the door to even more powerful concepts. You can explore more advanced concepts such as container networking, streamlining the management of complex applications with Docker Compose, and how to make your application data persistent using volumes.
-
Use same Dockerfile for Dev & Production
In many projects that are containerized, especially in cases where development is also done locally with docker-compose, teams often have two Dockerfiles, 1 for Development, the other for Production. If you happen to have multiple environments like pre-prod, staging and so on, some teams could have different Dockerfiles for these environments.
What are some alternatives?
Apache AGE - Graph database optimized for fast analysis and real-time data processing. It is provided as an extension to PostgreSQL.
supervisor - Supervisor process control system for Unix (supervisord)
ldbc_snb_bi - Reference implementations for the LDBC Social Network Benchmark's Business Intelligence (BI) workload
LibreNMS-docker - LibreNMS Docker image
benchgraph
terraform - Terraform enables you to safely and predictably create, change, and improve infrastructure. It is a source-available tool that codifies APIs into declarative configuration files that can be shared amongst team members, treated as code, edited, reviewed, and versioned.
arcadedb - ArcadeDB Multi-Model Database, one DBMS that supports SQL, Cypher, Gremlin, HTTP/JSON, MongoDB and Redis. ArcadeDB is a conceptual fork of OrientDB, the first Multi-Model DBMS. ArcadeDB supports Vector Embeddings.
Portainer - Making Docker and Kubernetes management easy.
simple-graph - This is a simple graph database in SQLite, inspired by "SQLite as a document database"
Cloud-Init - unofficial mirror of Ubuntu's cloud-init
nebula-docker-compose - Docker compose for Nebula Graph
k3s - Lightweight Kubernetes