dremio-oss
cockroach
dremio-oss | cockroach | |
---|---|---|
8 | 100 | |
1,301 | 29,119 | |
0.8% | 0.8% | |
4.0 | 10.0 | |
14 days ago | 3 days ago | |
Java | Go | |
Apache License 2.0 | GNU General Public License v3.0 or later |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
dremio-oss
-
What is the separation of storage and compute in data platforms and why does it matter?
Dremio - Dremio is a data lakehouse based on the open-source Apache Iceberg table format. It offers different compute instances to process data that lives in your S3 bucket. You pay for S3 storage independently.
-
What is dremio query engine
Dremio core is actually fully open source: https://github.com/dremio/dremio-oss
-
Q – Run SQL Directly on CSV or TSV Files
I have been using Dremio to query large volume of CSV files: https://docs.dremio.com/software/data-sources/files-and-dire...
Although having them in some columnar format is much better for fast responses.
GitHub: https://github.com/dremio/dremio-oss
-
Hands-On Introduction to Apache Iceberg - Data Lakehouse Engineering
As a Developer Advocate for Dremio I spend a lot of time doing research on technology and best practices around engineering Data Lakehouses and sharing what I learn through content for Subsurface - The Data Lakehouse Community. One of the major topics I've been diving deep into is the topic of Data Lakehouse Table Formats, these allow you to take the files on your data lake and group them into tables data processing engines like Dremio can operate on.
-
Introduction to The World of Data - (OLTP, OLAP, Data Warehouses, Data Lakes and more)
Hearing about all these components sounds great, but what everyone wants isn't to have to setup and configure all these components but instead have a platform and tool that brings this all together in an easy to use package, and that platform is Dremio. With Dremio you can work with the data directly from your data lake. No copies, easy access, high performance.
-
Data Lakehouse and Delta Lake
And as u/pych_phd said, it's not just Databricks, Snowflake and Azure who make these claims, even AWS, GCP, Dremio and I'm sure many others are too.
-
Data Science Competition
Dremio
-
Build your own “data lake” for reporting purposes
For my home projects I generate parquet (columnar and very well suited for DW like queries) files with pyarrow and use https://github.com/dremio/dremio-oss (https://www.dremio.com/on-prem/) to query them on lake (minio or just local disk or s3) and use Apache Superset for quick charts or dashboards.
cockroach
-
11 Planetscale alternatives with free tiers
CockroachDB is an open source distributed SQL database designed for scalability and resilience. While it offers SQL databases, CockroachDB is also compatible with PostgreSQL.
-
A MySQL compatible database engine written in pure Go
cockroachdb might be close: https://github.com/cockroachdb/cockroach
-
No More Free Tier on PlanetScale, Here Are Free Alternatives
CockroachDB - SQL
-
Is it bad to create a publicly accessible RDS database for my serverless web app?
For example, when you create a serverless postgres database with a platform like CockroachDB or Neon, you effectively get a connection string with a strong password. Anyone can connect to your database from anywhere so long as they have the right connection string. There are no security settings in these services to change this behavior.
-
Linux surpasses the Mac among Steam gamers
> Yes you can on the android emulator. The biggest issue is compu arch in that case.
I can also download VirtualBox and run all Windows programs, that would mean that all Windows apps are Linux apps?
> Yes you can for the most part
You can't statically link glibc: https://github.com/cockroachdb/cockroach/issues/3392
glibc can break stuff: https://www.gamingonlinux.com/2022/08/valve-dev-understandab...
I had binaries break because the newer version if openssl was put under a slightly different name.
-
How do small SaaS's handle databases?
Also, worth noting, if you're already using PostgreSQL (or plan to) you might want to take a look at https://www.cockroachlabs.com/ they have a free tier too and CockroachDB has a PostgreSQL interface.
-
Go Dependency management in large company projects - How do you do it?
I know that some projects like cockroach use custom build tools like bazel. But we actually really like to use to be able to build our projects simply with the great go toolchain and don't really aim to dive deep into custom build solutions.
-
Eli5: Why do companies use the products of Oracle to store information, when they can just use spreadsheets like Excel, or make their own spreadsheet software?
CockroachDB is designed to be globally distributed. It has to handle causality when resolving collisions. It has to account for having a write operation to arrive after another and still have time priority because it was sent out a few milliseconds earlier.
-
rage - a minimalistic load testing tool
Cockroachdb created a go runtime patch which measures the Grunning time of a goroutine: https://github.com/cockroachdb/cockroach/pull/82356. It doesn't entirely solve the problem though.
-
Data Engineering Tools in Go
Our entire backend is written in Go. We've built a platform that allows other companies to offer automatic data syncing to their customers' data warehouses. Go works great for building distributed systems like this (see K8s). We're not the only ones in the space building data intensive applications with Go. Pachyderm, Pinecone, Cockroach Labs and are all also doing it. We've been quite happy with how Go has worked for us.
What are some alternatives?
Trino - Official repository of Trino, the distributed SQL query engine for big data, formerly known as PrestoSQL (https://trino.io)
vitess - Vitess is a database clustering system for horizontal scaling of MySQL.
presto - Official repository of Trino, the distributed SQL query engine for big data, formerly known as PrestoSQL (https://trino.io) [Moved to: https://github.com/trinodb/trino]
neon - Neon: Serverless Postgres. We separated storage and compute to offer autoscaling, branching, and bottomless storage.
ClickHouse - ClickHouse® is a free analytics DBMS for big data
tidb - TiDB is an open-source, cloud-native, distributed, MySQL-Compatible database for elastic scale and real-time analytics. Try AI-powered Chat2Query free at : https://tidbcloud.com/free-trial
Greenplum - Greenplum Database - Massively Parallel PostgreSQL for Analytics. An open-source massively parallel data platform for analytics, machine learning and AI.
Grafana - The open and composable observability and data visualization platform. Visualize metrics, logs, and traces from multiple sources like Prometheus, Loki, Elasticsearch, InfluxDB, Postgres and many more.
yugabyte-db - YugabyteDB - the cloud native distributed SQL database for mission-critical applications.
Rakam - 📈 Collect customer event data from your apps. (Note that this project only includes the API collector, not the visualization platform)
InfluxDB - Scalable datastore for metrics, events, and real-time analytics