NebulaGraph Database VS apisix

A NoSQL graph database is a type of non-relational, distributed database which employs a graph model. NoSQL stands for “Not only SQL” and refers to a new breed of databases that differ from traditional relational databases in their data model and performance. Graph databases are especially useful for data associated with relationships—everything from friendships on social netwo#rks to equipment supply chains or business processes. They can quickly traverse vast amounts of linked data points to discover insights and hidden connections between entities, making them ideal for network analysis– such as financial fraud detection, recommendation engines and many other use cases– all while performing at scale.

An open source graph database is always the best place to start as they come with a supportive community that ultimately creates the perfect ecosystem.

When we first came across NebulaGraph, because the ecology was not perfect, and only some businesses migrated to Nebula, we used to import NebulaGraph data, whether full or incremental, by pushing Hive tables to Kafka and consuming Kafka to write NebulaGraph in batch. Later, as more and more data and businesses switched to NebulaGraph, the problem of importing data efficiency became more and more serious. The increase in import time made it unacceptable to still be importing data at full volume during peak business hours.

NebulaGraph NebulaGraph is a popular open-source graph database that can handle large volumes of data with milliseconds of latency, scale up quickly, and have the ability to perform fast graph analytics. Official website https://www.nebula-graph.io/ GitHub projects https://github.com/orgs/vesoft-inc/repositories https://github.com/vesoft-inc/nebula

NebulaGraph, a leading open source graph database, announced it raised tens of millions of US dollars in Series A funding. Investors in the round are led by Jeneration Capital, with participation from the previous investors - Matrix Partner China, Redpoint China Ventures, and Source Code Capital. China Renaissance served as the exclusive financial advisor in this financing round.

Support backup and restore. https://github.com/vesoft-inc/nebula/pull/3469 https://github.com/vesoft-inc/nebula-agent/pull/1 https://github.com/vesoft-inc/nebula-br/pull/22

One option is NebulaGraph if your looking for low latency, scalability, and HA.

After the step-by-step decomposition of the patterns, the expected execution plan for the MATCH clause is finally generated. As you can see, it takes a lot of effort to transform a complicated pattern into the underlying interfaces for a traversal. Of course, the execution plan can be optimized, such as the multi-step traversal can be encapsulated by using the Loop operator and the sub-plan of a one-step traversal can be reused, which will not be detailed in this article. If you are interested, please refer to the source code of Nebula Graph.

> I think etcd is basically a k8s only project now

I hate etcd with the best of them, but etcd is used in a lot more places than just kubernetes:

https://github.com/apache/apisix/blob/master/docs/en/latest/...

https://github.com/traefik/traefik#:~:text=Etcd,

https://github.com/zalando/patroni#patroni-a-template-for-po...

https://github.com/purpleidea/mgmt/tree/0.0.26/etcd (this one shows up on HN quite a bit)

https://github.com/sorintlab/stolon#features

It's actually one of the major reasons I wouldn't touch those projects

Through this intelligent caching mechanism, APISIX efficiently utilizes system resources when handling a large volume of requests, thereby improving overall system performance and stability. APISIX, with its advanced LRU cache, provides developers with a reliable and efficient API gateway solution, facilitating smooth communication with external services.

The main issue is that priority is documented in the config-default.yaml file, while the phase is buried in the code. Worse, some plugins run across different phases. For example, let's check the proxy proxy-rewrite plugin and, more precisely, the functions defined there:

// references: // https://github.com/tetratelabs/proxy-wasm-go-sdk/tree/main/examples // https://github.com/apache/apisix/blob/master/t/wasm/ package main import ( "github.com/tetratelabs/proxy-wasm-go-sdk/proxywasm" "github.com/tetratelabs/proxy-wasm-go-sdk/proxywasm/types" "github.com/valyala/fastjson" ) func main() { proxywasm.SetVMContext(&vmContext{}) } // each plugin has its own VMContext. // it is responsible for creating multiple PluginContexts for each route. type vmContext struct { types.DefaultVMContext } // each route has its own PluginContext. // it corresponds to one instance of the plugin. func (*vmContext) NewPluginContext(contextID uint32) types.PluginContext { return &pluginContext{} } type header struct { Name string Value string } type pluginContext struct { types.DefaultPluginContext Headers []header } func (ctx *pluginContext) OnPluginStart(pluginConfigurationSize int) types.OnPluginStartStatus { data, err := proxywasm.GetPluginConfiguration() if err != nil { proxywasm.LogErrorf("error reading plugin configuration: %v", err) return types.OnPluginStartStatusFailed } var p fastjson.Parser v, err := p.ParseBytes(data) if err != nil { proxywasm.LogErrorf("error decoding plugin configuration: %v", err) return types.OnPluginStartStatusFailed } headers := v.GetArray("headers") ctx.Headers = make([]header, len(headers)) for i, hdr := range headers { ctx.Headers[i] = header{ Name: string(hdr.GetStringBytes("name")), Value: string(hdr.GetStringBytes("value")), } } return types.OnPluginStartStatusOK } // each HTTP request to a route has its own HTTPContext func (ctx *pluginContext) NewHttpContext(contextID uint32) types.HttpContext { return &httpContext{parent: ctx} } type httpContext struct { types.DefaultHttpContext parent *pluginContext } func (ctx *httpContext) OnHttpResponseHeaders(numHeaders int, endOfStream bool) types.Action { plugin := ctx.parent for _, hdr := range plugin.Headers { proxywasm.ReplaceHttpResponseHeader(hdr.Name, hdr.Value) } return types.ActionContinue }

API7 takes Apache APISIX as its robust foundation, which is open-source and has an active community with over 600 contributors all over the world. The nature of open source allows users to examine the source code, which promotes transparency. This transparency helps users understand how APISIX works, verify its security, and identify and fix any potential vulnerabilities or bugs.

But the best part is that the libraries mentioned here and Apache APISIX are entirely open source, meaning you can look under the hood and modify things yourself.

Default configuration

4. Plugin definition: It is a really important part of plugin implementation that we define as a table with properties for the version, priority, name, and schema. The name and schema are the plugin's name and schema defined earlier. The version and priority are used by APISIX to manage the plugin. The version typically refers to the version that is currently in use like API versioning. If you publish and update your plugin logic, it is going to be 1.1 (You can set any version you wish). But you need to be very careful in choosing priority. The priority field defines in which order and phase your plugin should be executed. For example, the 'ip-restriction' plugin, with a priority of 3000, will be executed before the 'example-plugin', which has a priority of 0. This is due to the higher priority value of the 'ip-restriction' plugin. If you're developing your own plugin, make sure that you followed the order of plugins not to mess up the order of existing plugins. You can check the order of existing plugins in the config-default.yaml file and open the Apache APISIX Plugin Development Guide to determine.

Their use of etcd was a hard pass for me; I don't need more etcd in my life

Apache APISIX Apache APISIX is an open source, dynamic, real-time, high-performance cloud native API gateway. APISIX provides rich traffic management features such as load balancing, dynamic upstream, canary release, circuit breaking, authentication, observability, and more. Official website https://apisix.apache.org/ GitHub projects APISIX (the core): https://github.com/apache/apisix GitHub - apache/apisix: The Cloud-Native API Gateway GitHub - apache/apisix-dashboard: Dashboard for Apache APISIX GitHub - apache/apisix-website: Apache APISIX Website GitHub - apache/apisix-docker: the docker for Apache APISIX GitHub - apache/apisix-go-plugin-runner: Go Plugin Runner for APISIX GitHub - apache/apisix-java-plugin-runner: APISIX Plugin Runner in Java GitHub - apache/apisix-python-plugin-runner: Apache APISIX Python plugin runner GitHub - apache/apisix-helm-chart: Apache APISIX Helm Chart GitHub - apache/apisix-ingress-controller: ingress controller for K8s