opendal
sccache
opendal | sccache | |
---|---|---|
10 | 71 | |
2,858 | 5,365 | |
2.5% | 1.6% | |
9.9 | 9.4 | |
1 day ago | 8 days ago | |
Rust | Rust | |
Apache License 2.0 | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
opendal
-
Welcome to Apache OpenDAL
Sounds likely.
The core part of OpenDAL is a Rust crate that provides fs-like APIs over different storage backends, but we also investigate providing other interfaces like a CLI. We have an experimental binary named `oli`[1].
You're welcome to start a discussion[2] to share how you use rclone and we may find it fit in OpenDAL's scope :D
[1] https://github.com/apache/incubator-opendal/tree/main/bin/ol...
-
Hey Rustaceans! Got a question? Ask here (49/2023)!
[profiles.mys3] type = "s3" region = "us-east-1" access_key_id = "foo" enable_virtual_host_style = "on" ``` The team at Opendal wrote a handcrafting config parser for the same use case, see. Since parsing configs in toml or json is a standard functionality, is there any recommended way?
-
Ask HN: Experience using your user's Google Drive instead of a database?
I've often felt we need an abstraction for just this. "Bring your own storage" so that you can sign up and provide a "bucket", then the service will write to that.
OpenDAL was on HN recently and would be a pretty decent abstraction to use for this: https://github.com/apache/incubator-opendal
-
Rust std:fs slower than Python
Totally unrelated but: this post talks about the bug being first discovered in OpenDAL [1], which seems to be an Apache (Incubator) project to add an abstraction layer for storage over several types of storage backend. What's the point/use case of such an abstraction? Anybody using it?
[1] https://opendal.apache.org/
-
S3 as the Storage Layer
https://github.com/apache/incubator-opendal
- Apache OpenDAL: A unified data access layer
- Apache OpenDAL
-
Way to Go: OpenDAL successfully entered Apache Incubator
A new big event in a few weeks, this may be the first project whose primary language is Rust to enter the Apache incubator. OpenDAL originated from the vision of creating a universal, unified and user-friendly data access layer. It came into being in late 2021, initially as a component of the Databend project.
-
[Need inspiration] Building the control plane of a search engine (Quickwit)
I was reading through the code of databend: https://databend.rs/ It's a "wrapper" over datafusion and does a lot of similar things to Quickwit. And yeah, to drive the index cluster they rely on https://github.com/datafuselabs/openraft && https://github.com/datafuselabs/opendal. I'd be interested about you thoughts on the project if you've already heard about it too.
-
[Announcement] Databend v0.7.0 Released!
Announce OpenDAL for object storage data access
sccache
-
Speeding up C++ build times
Use icecream or sccache. sccache supports distributed builds.
https://github.com/mozilla/sccache/blob/main/docs/Distribute...
-
Mozilla sccache: cache with cloud storage
Worth noting that the first commit in sccache git repository was in 2014 (https://github.com/mozilla/sccache/commit/115016e0a83b290dc2...). So I suppose that what "happened" happened waay back.
- Welcome to Apache OpenDAL
-
Target file are very huge and running out of storage on mac.
If you have lots of shared dependencies, maybe try sccache?
-
S3 Express Is All You Need
I'm going to set up sccache [0] to use it tomorrow. We use MSVC, so EFS is off the cards.
[0] https://github.com/mozilla/sccache/blob/main/docs/S3.md
- sccache
-
Serde has started shipping precompiled binaries with no way to opt out
I think the primary benefit of pre-built procmacros will be for build servers which don't use a persistent cache (like sccache), since they have to compile all dependencies every time. But IMO improved support for persistent caches would be a better investment compared to adding support for pre-built procmacros.
-
Cache dependencies across crates
Checkout https://github.com/mozilla/sccache
-
Distcc: A fast, free distributed C/C++ compiler
https://github.com/mozilla/sccache is another option which addresses the use cases of both icecream and ccache (and also supports Rust, and cloud storage of artifacts, if those are useful for you)
-
How to fix Rust Coding LARGE files????
That being said a compilation cache, eg the de-facto standard for Rust: sccache (https://github.com/mozilla/sccache) will help to compile and store some of the build artifacts centralized - still for each crate version + build profile (RUSTFLAGS) combination.
What are some alternatives?
databend - 𝗗𝗮𝘁𝗮, 𝗔𝗻𝗮𝗹𝘆𝘁𝗶𝗰𝘀 & 𝗔𝗜. Modern alternative to Snowflake. Cost-effective and simple for massive-scale analytics. https://databend.com
ccache - ccache – a fast compiler cache
openraft - rust raft with improvements
cargo-chef - A cargo-subcommand to speed up Rust Docker builds using Docker layer caching.
s3s - S3 Service Adapter
rust-cache - A GitHub Action that implements smart caching for rust/cargo projects
fluvio - Lean and mean distributed stream processing system written in rust and web assembly.
cache - Cache dependencies and build outputs in GitHub Actions
dilbert-viewer - A simple comic viewer for Dilbert by Scott Adams
icecream - Distributed compiler with a central scheduler to share build load
storage - A vendor-neutral storage library for Golang: Write once, run on every storage service.
mold - Mold: A Modern Linker 🦠