automq VS fury-benchmarks

[1] Cloud-Init: https://github.com/canonical/Cloud-Init [2] AutoMQ: https://github.com/AutoMQ/automq [3] Introduction to Cloud-Init: https://cloudinit.readthedocs.io/en/latest/explanation/introduction.html#how-does-Cloud-Init-work

AutoMQ: 1.1.0 https://github.com/AutoMQ/automq/releases/tag/1.1.0-rc0

References [1] Cloud Disks are (Really!) Expensive: https://www.warpstream.com/blog/cloud-disks-are-expensive [2] Making Apache Kafka Serverless: Lessons From Confluent Cloud: https://www.confluent.io/blog/designing-an-elastic-apache-Kafka-for-the-cloud/#self-balancing-clusters [3] How AutoMQ addresses the disk read side effects in Apache Kafka: https://www.automq.com/blog/how-automq-addresses-the-disk-read-side-effects-in-apache-Kafka [4] Broker performance degradation caused by call of sendfile reading disk in network thread:https://issues.apache.org/jira/browse/Kafka-7504 [5] Is there anyway to activate auto scaling or some form of auto scaling with Strimzi? : https://github.com/orgs/strimzi/discussions/6635 [6] Introducing Confluent Cloud Freight Clusters: https://www.confluent.io/blog/introducing-confluent-cloud-freight-clusters/ [7] Resize a Dedicated Kafka Cluster in Confluent Cloud: https://docs.confluent.io/cloud/current/clusters/resize.html [8] Why is low latency important?:https://redpanda.com/guides/Kafka-performance/Kafka-latency [9] Public Benchmarks and TCO Analysis: https://www.warpstream.com/blog/warpstream-benchmarks-and-tco [10] AutoMQ: https://github.com/AutoMQ/automq [11] AWS EBS Multi-Attach: https://docs.aws.amazon.com/ebs/latest/userguide/ebs-volumes-multi.html [12] NVME reservations: https://docs.aws.amazon.com/ebs/latest/userguide/nvme-reservations.html [13] Nitro card for Amazon EBS: https://d1.awsstatic.com/events/Summits/reinvent2023/STG210_Behind-the-scenes-of-Amazon-EBS-innovation-and-operational-excellence.pdf [14] How AutoMQ addresses the disk read side effects in Apache Kafka: https://www.automq.com/blog/how-automq-addresses-the-disk-read-side-effects-in-apache-Kafka [15] Kafka is dead, long live Kafka: https://www.warpstream.com/blog/Kafka-is-dead-long-live-Kafka [16] GCP Regional Persistent Disk: https://cloud.google.com/compute/docs/disks/high-availability-regional-persistent-disk [17] Azure ZRS Disk: https://learn.microsoft.com/en-us/azure/virtual-machines/disks-deploy-zrs?tabs=portal

I agree with your viewpoint. The crux of the matter is not whether to use tiered storage or not, but what trade-offs have been made in the specific storage architecture and what benefits have been gained. Here(https://github.com/AutoMQ/automq?tab=readme-ov-file#-automq-...) is a qualitative comparison chart of streaming systems including kafka/confluent/redpanda/warpstream/automq. This comparison chart does not have specific numerical comparisons, but purely based on their trade-offs at the storage level, I think this will be of some use to you.

1) Fury is 41.6x faster than jackson for Struct serialization 2) Fury is 65.6x faster than jackson for Struct deserialization 3) Fury is 9.4x faster than jackson for MediaContent serialization 4) Fury is 9.6x faster than jackson for MediaContent deserialization

see https://github.com/chaokunyang/fury-benchmarks for detailed benchmark code.