ldbc_snb_bi
Apache AGE
ldbc_snb_bi | Apache AGE | |
---|---|---|
3 | 133 | |
33 | 2,753 | |
- | 5.6% | |
7.7 | 8.9 | |
4 months ago | 4 days ago | |
Python | C | |
Apache License 2.0 | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
ldbc_snb_bi
-
Demand the impossible: rigorous database benchmarking
Rigorous database benchmarking is indeed very difficult and time-consuming. I spent the last ~7 years working on benchmarks for graph processing systems in the Linked Data Benchmark Council (LDBC) [1], originally established in 2012 as an EU research project.
LDBC creates TPC-style application-level database benchmarks which can be used for system-to-system comparison. We provide detailed specifications, data generators, benchmark frameworks, and multiple reference implementations. The benchmarks are implemented by vendors for their database products, and the implementations submitted to be run by independent third-party auditors to ensure their correctness and reproducibility.
We have found that there is a market for audits for graph processing systems, albeit it is quite small: over the last 4 years, we have published 34 audited results, see e.g. [2] and [3].
A major problem we face is that process of implementing the benchmark for a system and getting an audited result is long (and therefore expensive). Vendors spend months implementing the and tuning the benchmarks. It is also typical for the auditor to spend 50+ hours on the auditing process, which includes a lengthy code review step, setting up the system, running the experiments, testing ACID properties, writing a report, etc. The length of the process is exacerbated by the lack of standard graph query languages. This potentially necessitates the auditor to learn a new query language or programming language.
We have tried to mitigate this problem by improving our documentation, creating more reference implementation, distributing pre-generated data sets. There are new standard graph query languages (SQL/PGQ, GQL) but their adoption is still very limited. Overall, the auditing process is quite long, which is mainly caused by the essential complexity of the problem: implementing an application-level benchmark and getting reliable results is very difficult.
[1] https://ldbcouncil.org/introduction/
[2] https://ldbcouncil.org/benchmarks/snb-interactive
[3] https://ldbcouncil.org/benchmarks/snb-bi/
-
Benchgraph Backstory: The Untapped Potential
At first, the plan was to use only the LDBC dataset and write different queries for the dataset, but LDBC has a set of well-designed queries that were specifically prepared to stress the database. Each query targets a special scenario, also called “chock point.” Not to be mistaken, they do not have deep graph traversal doing around 100 hops, but they are definitely more complex than the ones written for the Pokec dataset. There are two sets of queries for the LDBC SNB: interactive and business intelligence. LDBC provides a reference Cypher implementation for both of these queries for Neo4j. We took those queries, tweaked the data types, and made the queries work on Memgraph. Again, to be perfectly clear, this is NOT an official implementation of an LDBC Benchmark; this goes for both interactive and business intelligence queries. The queries were used as the basis for running the benchmark.
-
Postgres: The Graph Database You Didn't Know You Had
I designed and maintain several graph benchmarks in the Linked Data Benchmark Council, including workloads aimed for databases [1]. We make no restrictions on implementations, they can any query language like Cypher, SQL, etc.
In our last benchmark aimed at analytical systems [2], we found that SQL queries using WITH RECURSIVE can work for expressing reachability and even weighted shortest path queries. However, formulating an efficient algorithm yields very complex SQL queries [3] and their execution requires a system with a sophisticated optimizer such as Umbra developed at TU Munich [4]. Industry SQL systems are not yet at this level but they may attain that sometime in the future.
Another direction to include graph queries in SQL is the upcoming SQL/PGQ (Property Graph Queries) extension. I'm involved in a project at CWI Amsterdam to incorporate this language into DuckDB [5].
[1] https://ldbcouncil.org/benchmarks/snb/
[2] https://www.vldb.org/pvldb/vol16/p877-szarnyas.pdf
[3] https://github.com/ldbc/ldbc_snb_bi/blob/main/umbra/queries/...
[4] https://umbra-db.com/
[5] https://www.cidrdb.org/cidr2023/slides/p66-wolde-slides.pdf
Apache AGE
- Apache AGE: PostgreSQL Extension Graph Data Processing and Analytics
- Apache AGE: PostgreSQL Extension Graph Data Processing and Analytics for RDBMS
- Apache AGE supporting latest PostgreSQL (ver 16)
-
Enhancing Fraud Detection with Apache AGE: A Graph Database Approach
For more information and support, visit the Apache AGE website. or github.
-
Unlocking the Power of Apache Age: Advanced Techniques for SQL/Cypher Hybrid Queries
These are just a few examples of how you can use Cypher queries in SQL/Cypher Hybrid Queries. By using these advanced techniques, you can perform more complex and powerful queries on your graph data. Apache AGE offer a versatile and powerful platform for working with graph and relational data concurrentlyz. To learn more you can visit age website or github page.
-
Mastering Graph Queries with Cypher in Apache Age
Cypher queries in Apache Age empower users to interact with graph data efficiently and intuitively. Whether you're creating nodes, establishing relationships, or performing complex traversals, Cypher provides a robust and expressive language for working with graph databases.As you explore Apache Age and Cypher further, you'll discover additional features and nuances that make graph database management a seamless experience. Embrace the power of graph queries, and unlock the full potential of your interconnected data with Apache Age. Happy graph querying!
- We built An Open-Source platform to process relational and Graph Query simultaneously
-
Machine learning and graph databases
Check Apache AGE graph database system here: Website: https://age.apache.org/ GitHub: https://github.com/apache/age
- Is Open Sourcing Technologies Good for Society
-
Open Source doesn't win by being cheaper
We are also open Source community at Apache https://github.com/apache/age
What are some alternatives?
ldbc_snb_datagen_spark - Synthetic graph generator for the LDBC Social Network Benchmark, running on Spark
surrealdb - A scalable, distributed, collaborative, document-graph database, for the realtime web
spicedb - Open Source, Google Zanzibar-inspired permissions database to enable fine-grained access control for customer applications
Memgraph - Open-source graph database, tuned for dynamic analytics environments. Easy to adopt, scale and own.
ldbc_snb_interactive_v1_impls - Reference implementations for LDBC Social Network Benchmark's Interactive workload.
node-bindgen - Easy way to write Node.js module using Rust
materialize - The data warehouse for operational workloads.
age-viewer - Graph database optimized for fast analysis and real-time data processing. It is provided as an extension to PostgreSQL.
clair - Vulnerability Static Analysis for Containers
napi-rs - A framework for building compiled Node.js add-ons in Rust via Node-API
quine - Quine • a streaming graph • https://quine.io • Discord: https://discord.gg/GMhd8TE4MR
neon - Rust bindings for writing safe and fast native Node.js modules.