druid-datasets VS cockroach

Apache NiFi supports powerful and scalable directed graphs of data routing, transformation, and system mediation logic. Nifi is very useful when data needs to be loaded from different sources. In this case, I will nifi to access the Github API as it is very easy to make repeated calls to a Http endpoint and get data from multiple pages. You can see what I did by downloading NiFi yourself and then adding my template from the Druid Datasets repo: https://github.com/implydata/druid-datasets/blob/main/githubstars/github_stars.xml

For example, when you create a serverless postgres database with a platform like CockroachDB or Neon, you effectively get a connection string with a strong password. Anyone can connect to your database from anywhere so long as they have the right connection string. There are no security settings in these services to change this behavior.

> Yes you can on the android emulator. The biggest issue is compu arch in that case.

I can also download VirtualBox and run all Windows programs, that would mean that all Windows apps are Linux apps?

> Yes you can for the most part

You can't statically link glibc: https://github.com/cockroachdb/cockroach/issues/3392

glibc can break stuff: https://www.gamingonlinux.com/2022/08/valve-dev-understandab...

I had binaries break because the newer version if openssl was put under a slightly different name.

Also, worth noting, if you're already using PostgreSQL (or plan to) you might want to take a look at https://www.cockroachlabs.com/ they have a free tier too and CockroachDB has a PostgreSQL interface.

I know that some projects like cockroach use custom build tools like bazel. But we actually really like to use to be able to build our projects simply with the great go toolchain and don't really aim to dive deep into custom build solutions.

Cockroachdb created a go runtime patch which measures the Grunning time of a goroutine: https://github.com/cockroachdb/cockroach/pull/82356. It doesn't entirely solve the problem though.

Our entire backend is written in Go. We've built a platform that allows other companies to offer automatic data syncing to their customers' data warehouses. Go works great for building distributed systems like this (see K8s). We're not the only ones in the space building data intensive applications with Go. Pachyderm, Pinecone, Cockroach Labs and are all also doing it. We've been quite happy with how Go has worked for us.

I would recommend researching CockroachDB. It does what you're describing natively. The basis of it's design is to distribute data based on geography. It's inexpensive and very simple to install and manage. It's based on Postgres ( but doesn't support all Postgres features). We've been using it for about 1 year and it meets our needs for some of our workload. www.cockroachlabs.com

CockroachDB

You absolutely can write very high performance software in Go, that's kind of the point. You can efficiently interface with C libraries. You can create the sort of software everyone says should be done in Rust, like databases and web servers and system orchestration and games and every other goddamn thing that people will say isn't the right choice for Go.