opendal
julia
opendal | julia | |
---|---|---|
10 | 350 | |
2,858 | 44,569 | |
2.5% | 0.6% | |
9.9 | 10.0 | |
1 day ago | 3 days ago | |
Rust | Julia | |
Apache License 2.0 | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
opendal
-
Welcome to Apache OpenDAL
Sounds likely.
The core part of OpenDAL is a Rust crate that provides fs-like APIs over different storage backends, but we also investigate providing other interfaces like a CLI. We have an experimental binary named `oli`[1].
You're welcome to start a discussion[2] to share how you use rclone and we may find it fit in OpenDAL's scope :D
[1] https://github.com/apache/incubator-opendal/tree/main/bin/ol...
-
Hey Rustaceans! Got a question? Ask here (49/2023)!
[profiles.mys3] type = "s3" region = "us-east-1" access_key_id = "foo" enable_virtual_host_style = "on" ``` The team at Opendal wrote a handcrafting config parser for the same use case, see. Since parsing configs in toml or json is a standard functionality, is there any recommended way?
-
Ask HN: Experience using your user's Google Drive instead of a database?
I've often felt we need an abstraction for just this. "Bring your own storage" so that you can sign up and provide a "bucket", then the service will write to that.
OpenDAL was on HN recently and would be a pretty decent abstraction to use for this: https://github.com/apache/incubator-opendal
-
Rust std:fs slower than Python
Totally unrelated but: this post talks about the bug being first discovered in OpenDAL [1], which seems to be an Apache (Incubator) project to add an abstraction layer for storage over several types of storage backend. What's the point/use case of such an abstraction? Anybody using it?
[1] https://opendal.apache.org/
-
S3 as the Storage Layer
https://github.com/apache/incubator-opendal
- Apache OpenDAL: A unified data access layer
- Apache OpenDAL
-
Way to Go: OpenDAL successfully entered Apache Incubator
A new big event in a few weeks, this may be the first project whose primary language is Rust to enter the Apache incubator. OpenDAL originated from the vision of creating a universal, unified and user-friendly data access layer. It came into being in late 2021, initially as a component of the Databend project.
-
[Need inspiration] Building the control plane of a search engine (Quickwit)
I was reading through the code of databend: https://databend.rs/ It's a "wrapper" over datafusion and does a lot of similar things to Quickwit. And yeah, to drive the index cluster they rely on https://github.com/datafuselabs/openraft && https://github.com/datafuselabs/opendal. I'd be interested about you thoughts on the project if you've already heard about it too.
-
[Announcement] Databend v0.7.0 Released!
Announce OpenDAL for object storage data access
julia
-
Top Paying Programming Technologies 2024
34. Julia - $74,963
-
Optimize sgemm on RISC-V platform
I don't believe there is any official documentation on this, but https://github.com/JuliaLang/julia/pull/49430 for example added prefetching to the marking phase of a GC which saw speedups on x86, but not on M1.
-
Dart 3.3
3. dispatch on all the arguments
the first solution is clean, but people really like dispatch.
the second makes calling functions in the function call syntax weird, because the first argument is privileged semantically but not syntactically.
the third makes calling functions in the method call syntax weird because the first argument is privileged syntactically but not semantically.
the closest things to this i can think of off the top of my head in remotely popular programming languages are: nim, lisp dialects, and julia.
nim navigates the dispatch conundrum by providing different ways to define free functions for different dispatch-ness. the tutorial gives a good overview: https://nim-lang.org/docs/tut2.html
lisps of course lack UFCS.
see here for a discussion on the lack of UFCS in julia: https://github.com/JuliaLang/julia/issues/31779
so to sum up the answer to the original question: because it's only obvious how to make it nice and tidy like you're wanting if you sacrifice function dispatch, which is ubiquitous for good reason!
-
Julia 1.10 Highlights
https://github.com/JuliaLang/julia/blob/release-1.10/NEWS.md
-
Best Programming languages for Data Analysis๐
Visit official site: https://julialang.org/
-
Potential of the Julia programming language for high energy physics computing
No. It runs natively on ARM.
julia> versioninfo() Julia Version 1.9.3 Commit bed2cd540a1 (2023-08-24 14:43 UTC) Build Info: Official https://julialang.org/ release
-
Rust std:fs slower than Python
https://github.com/JuliaLang/julia/issues/51086#issuecomment...
So while this "fixes" the issue, it'll introduce a confusing time delay between you freeing the memory and you observing that in `htop`.
But according to https://jemalloc.net/jemalloc.3.html you can set `opt.muzzy_decay_ms = 0` to remove the delay.
Still, the musl author has some reservations against making `jemalloc` the default:
https://www.openwall.com/lists/musl/2018/04/23/2
> It's got serious bloat problems, problems with undermining ASLR, and is optimized pretty much only for being as fast as possible without caring how much memory you use.
With the above-mentioned tunables, this should be mitigated to some extent, but the general "theme" (focusing on e.g. performance vs memory usage) will likely still mean "it's a tradeoff" or "it's no tradeoff, but only if you set tunables to what you need".
-
Eleven strategies for making reproducible research the norm
I have asked about Julia's reproducibility story on the Guix mailing list in the past, and at the time Simon Tournier didn't think it was promising. I seem to recall Julia itself didnt have a reproducible build. All I know now is that github issue is still not closed.
https://github.com/JuliaLang/julia/issues/34753
-
Julia as a unifying end-to-end workflow language on the Frontier exascale system
I don't really know what kind of rebuttal you're looking for, but I will link my HN comments from when this was first posted for some thoughts: https://news.ycombinator.com/item?id=31396861#31398796. As I said, in the linked post, I'm quite skeptical of the business of trying to assess relative buginess of programming in different systems, because that has strong dependencies on what you consider core vs packages and what exactly you're trying to do.
However, bugs in general suck and we've been thinking a fair bit about what additional tooling the language could provide to help people avoid the classes of bugs that Yuri encountered in the post.
The biggest class of problems in the blog post, is that it's pretty clear that `@inbounds` (and I will extend this to `@assume_effects`, even though that wasn't around when Yuri wrote his post) is problematic, because it's too hard to write. My proposal for what to do instead is at https://github.com/JuliaLang/julia/pull/50641.
Another common theme is that while Julia is great at composition, it's not clear what's expected to work and what isn't, because the interfaces are informal and not checked. This is a hard design problem, because it's quite close to the reasons why Julia works well. My current thoughts on that are here: https://github.com/Keno/InterfaceSpecs.jl but there's other proposals also.
-
Getaddrinfo() on glibc calls getenv(), oh boy
Doesn't musl have the same issue? https://github.com/JuliaLang/julia/issues/34726#issuecomment...
I also wonder about OSX's libc. Newer versions seem to have some sort of locking https://github.com/apple-open-source-mirror/Libc/blob/master...
but older versions (from 10.9) don't have any lockign: https://github.com/apple-oss-distributions/Libc/blob/Libc-99...
What are some alternatives?
databend - ๐๐ฎ๐๐ฎ, ๐๐ป๐ฎ๐น๐๐๐ถ๐ฐ๐ & ๐๐. Modern alternative to Snowflake. Cost-effective and simple for massive-scale analytics. https://databend.com
jax - Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more
openraft - rust raft with improvements
NetworkX - Network Analysis in Python
s3s - S3 Service Adapter
Lua - Lua is a powerful, efficient, lightweight, embeddable scripting language. It supports procedural programming, object-oriented programming, functional programming, data-driven programming, and data description.
fluvio - Lean and mean distributed stream processing system written in rust and web assembly.
rust-numpy - PyO3-based Rust bindings of the NumPy C-API
dilbert-viewer - A simple comic viewer for Dilbert by Scott Adams
Numba - NumPy aware dynamic Python compiler using LLVM
storage - A vendor-neutral storage library for Golang: Write once, run on every storage service.
F# - Please file issues or pull requests here: https://github.com/dotnet/fsharp