Delta-rs Alternatives
Similar projects and alternatives to delta-rs
-
delta
An open-source storage layer that brings scalable, ACID transactions to Apache Spark™ and big data workloads. (by delta-io)
-
Scout APM
Less time debugging, more time building. Scout APM allows you to find and fix performance issues with no hassle. Now with error monitoring and external services monitoring, Scout is a developer's best friend when it comes to application development.
-
materialize
Materialize simplifies application development with streaming data. Incrementally-updated materialized views - in PostgreSQL and in real time. Materialize is powered by Timely Dataflow. (by MaterializeInc)
-
-
roapi
Create full-fledged APIs for static datasets without writing a single line of code.
-
-
ballista
Distributed compute platform implemented in Rust, and powered by Apache Arrow.
-
OPS
OPS - Build and Run Open Source Unikernels. Quickly and easily build and deploy open source unikernels in tens of seconds. Deploy in any language to any cloud.
-
kafka-delta-ingest
A highly efficient daemon for streaming data from Kafka into Delta Lake
-
Reviews and mentions
-
Tipsy ramblings on Databricks' "Lakehouse"
Not sure if OP has come across this python library https://github.com/delta-io/delta-rs. With this library you can read delta tables stored in buckets and put them in pandas dataframe without all the Spark hassles.
-
Parquet or Orc?
If you want to use pure python it does not work yet: https://github.com/delta-io/delta-rs
- Delta-RS: Low level access to Delta tables in Rust
-
Rust for DE?
Speaking of Python, Rust has really good library and toolings (pyo03, maturin) to help write native Python modules. For example, we are able to create a Python binding for delta-rs with very little code: https://github.com/delta-io/delta-rs/tree/main/python.
We recently on boarded two more engineers onto https://github.com/delta-io/delta-rs and https://github.com/delta-io/kafka-delta-ingest. It took them about 1 week of training to be productive with Rust. So I wouldn't be too concerned about hiring. We never explicitly hired for Rust developers, all our Rust developers learned the language on the fly.
-
Delta Lake essential Fundamentals: Part 1 - ACID
delta-rs - Rust library for binding with Python and Ruby.
-
Databricks Is an RDBMS
> You cannot browse the data structure without an active cluster
That's one of the reason's I'm interested in delta-rs [1], which has delta lake bindings for Python. Would love to read a delta lake table into a native python object without the need for spark.
-
Intentionally Leaking AWS Keys
This looks to be the integration test: https://github.com/delta-io/delta-rs/commit/09d345e52ca3177d...
I'm not sure why the author thinks this required putting the credentials in the code itself. Heck, I'd argue that this integration test should be setting up the bucket and contents as part of the setup for the test itself, pulling the credentials from some sort of secrets manager.
Stats
delta-io/delta-rs is an open source project licensed under Apache License 2.0 which is an OSI approved license.
Popular Comparisons
Are you hiring? Post a new remote job listing for free.