s3-benchmark
rpCheckup
| s3-benchmark | rpCheckup | |
|---|---|---|
| 4 | 7 | |
| 776 | 157 | |
| - | -0.6% | |
| 0.0 | 0.0 | |
| 4 months ago | about 3 years ago | |
| Go | Go | |
| MIT License | Mozilla Public License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
s3-benchmark
- S3 Benchmark: Measure Amazon S3's performance from any location
- S3 Benchmark
-
Ask HN: Have you ever switched cloud?
There's another benchmark somewhere showing S3 can max out a 100Gbps instance.
https://github.com/dvassallo/s3-benchmark
Another potential issue is ListBucket rate limiting. If you have lots of small objects, you'll spend most of the time waiting to discover the names than transferring data
-
A distributed Posix file system built on top of Redis and S3
TTFB in S3 is 20-30ms around the 50th percentile. it can go much higher at p99 [1]. In any case, rotational latency for HDD drives is an order of magnitude lower (typically 2-5ms for a seek operation).
S3 is great for higher throughput workloads where TTFB is amortized across larger downloads (this is why it's very common to use S3 as a "data lake" where larger columnar files are stored, usually at the order of hundreds of MiB).
I think it's an interesting project but perhaps explaining the use cases where this solution is beneficial would go a long way here.
[1] https://github.com/dvassallo/s3-benchmark
rpCheckup
-
AWS GitHub script thoughts
What do people think of this? https://github.com/goldfiglabs/rpCheckup I am in no way affiliated with this - frankly I am worried if i use this ^ it might have other access things in my account? But i think all it needs is read only?
- RpCheckup – Catch AWS resource policy backdoors like Endgame
- Show HN: RpCheckup – Catch AWS resource policy backdoors like Endgame
- Open source: Catch AWS resource policy backdoors
- rpCheckup - Catch AWS resource policy backdoors like Endgame
What are some alternatives?
warp - S3 benchmarking tool
ScoutSuite - Multi-Cloud Security Auditing Tool
s5cmd - Parallel S3 and local filesystem execution tool.
ec2-macos-init - EC2 macOS Init is the launch daemon used to initialize Mac instances within EC2.
juicefs - JuiceFS is a distributed POSIX file system built on top of Redis and S3.
cloudquery - The open source high performance ELT framework powered by Apache Arrow
awesome-go-storage - A curated list of awesome Go storage projects and libraries
terrascan - Detect compliance and security violations across Infrastructure as Code to mitigate risk before provisioning cloud native infrastructure.
redisraft - A Redis Module that make it possible to create a consistent Raft cluster from multiple Redis instances.
cloudctl - CloudCtl is a GO library that interacts with cloud providers and displays output in a human-readable fashion.
containers-roadmap - This is the public roadmap for AWS container services (ECS, ECR, Fargate, and EKS).
introspector - A schema and set of tools for using SQL to query cloud infrastructure.