Our great sponsors
-
Object stores shard workloads based on an object name prefix, so if all log segments have the same prefix, they will hit the same storage server. This will lead to throttling and limit the upload throughput. To ensure good upload performance, Redpanda inserts a randomized prefix into every object. The prefix is computed using a xxHash hash function.
-
The development of the Shadow Indexing subsystem started with the write path. We developed our own S3 client using Seastar. However, Seastar didn’t allow us to use the existing object store client efficiently, and the framework didn’t have an http client that could be used to access the object store API. In order to overcome this challenge, we developed our own http client using Seastar.
-
WorkOS
The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
-
redpanda
Redpanda is a streaming data platform for developers. Kafka API compatible. 10x faster. No ZooKeeper. No JVM!
One of the key premises of Redpanda is unification of real-time and historical data by giving users the ability to store infinite data. However, in the modern cloud, the price of storage often dominates the price of all computing resources. The cost of object storage is vastly lower than the cost of the local disk attached to a compute node. Furthermore, object storage is often more reliable than the nodes themselves. To these ends, we created Shadow Indexing, a capability (available as a tech preview as of this writing) that allows us to capitalize on the 99.995% guaranteed reliability of the tier-4 datacenter.
