s3-benchmark
awesome-go-storage
s3-benchmark | awesome-go-storage | |
---|---|---|
4 | 7 | |
776 | 4,280 | |
- | 0.9% | |
0.0 | 4.1 | |
4 months ago | 4 months ago | |
Go | ||
MIT License | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
s3-benchmark
- S3 Benchmark: Measure Amazon S3's performance from any location
- S3 Benchmark
-
Ask HN: Have you ever switched cloud?
There's another benchmark somewhere showing S3 can max out a 100Gbps instance.
https://github.com/dvassallo/s3-benchmark
Another potential issue is ListBucket rate limiting. If you have lots of small objects, you'll spend most of the time waiting to discover the names than transferring data
-
A distributed Posix file system built on top of Redis and S3
TTFB in S3 is 20-30ms around the 50th percentile. it can go much higher at p99 [1]. In any case, rotational latency for HDD drives is an order of magnitude lower (typically 2-5ms for a seek operation).
S3 is great for higher throughput workloads where TTFB is amortized across larger downloads (this is why it's very common to use S3 as a "data lake" where larger columnar files are stored, usually at the order of hundreds of MiB).
I think it's an interesting project but perhaps explaining the use cases where this solution is beneficial would go a long way here.
[1] https://github.com/dvassallo/s3-benchmark
awesome-go-storage
-
Building a Log-Structured Merge Tree in Go
Awesome Go Storage (GitHub)
-
Open Source Databases in Go
Any many many more. Check https://github.com/gostor/awesome-go-storage
-
Zig, Hare, Odin, Vale, V, Jai
C is significantly slower at concurrency when implemented naively. It's as fast as languages like Go when implemented using the same techniques, which is not obvious and trivial to use like in a higher level GC'd language. GC actually helps out a ton there, for example look at the complexity of async/await in Rust which requires the notion of pinning.
https://github.com/gostor/awesome-go-storage#database
https://java-source.net/open-source/database-engines
Not a database but honorable mention, LMAX disrupter: https://lmax-exchange.github.io/disruptor/
- Embedded database options
-
Which database do you recommend to be used with Golang?
You may want to start from here: awesome-go-storage and choose what fit your needs
- New Open Source RDBMS idea (written in Golang) (Help wanted)
-
A distributed Posix file system built on top of Redis and S3
This is neat! I am quite a fan of all the go based file systems that are springing up. Question: what are the main compare and contrast points between juice and seaweed fs?
Here is a compendium for those interested:
https://github.com/gostor/awesome-go-storage
What are some alternatives?
warp - S3 benchmarking tool
chai - Modern embedded SQL database