Our great sponsors
-
-
-
WorkOS
The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
-
-
-
-
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
-
-
Centrifugo
Scalable real-time messaging server in a language-agnostic way. Self-hosted alternative to Pubnub, Pusher, Ably. Set up once and forever.
> What does "on the edge" mean in this context?
DriftDB has a concept of “rooms”, which are essentially broadcast channels. By “on the edge”, what I mean is that the authoritative server for each room can be geographically located near the participants in that room. In practice, today that means that it can be compiled to WebAssembly and run as a Cloudflare Worker.
> Can I just run the server part on my own infrastructure?
Kinda. It includes a server that runs locally, but it’s only useful as a development server at this point. Your question about multiple pods is exactly the reason -- unless you have a routing layer that is aware of DriftDB’s “rooms”, it won’t work if you scale it up. We also make https://plane.dev which provides the routing layer, but it might be overkill for a DriftDB use case.
Good feedback, here you go :) https://github.com/drifting-in-space/driftdb/commit/8d946217...
Re. #1, can you elaborate? The tracking issue for this is still open[1].
Re #2, the problem is that a rollup of a subject in NATS rolls up the whole subject, so there’s a race condition if you try to use it the way DriftDB uses it. If one client is computing a compaction while another client sends a message, that message will be erased by the compaction.
DriftDB solves this by sending a sequence number alongside the rollup of the last message included in the rollup, and the server preserves messages after that sequence number.
[1] https://github.com/nats-io/nats-server/issues/2667
Nats is not something I see as a competitor for external clients (browsers, mobile apps), primarily because it doesn't handle reconnections / message delivery / quality-of-service / at-least-once or exactly-once delivery (except for MQTT).
> When the connection is lost, your application would have to re-create it and all subscriptions if any. https://github.com/nats-io/stan.go#connection-status
Therefore, I don't see what it adds here. It seems designed for service communication, not client-server. They also don't list browsers as a use case https://docs.nats.io/nats-concepts/overview#use-cases. (though it is of course possible, it's just not ideal IMHO.)
They still have a js/browser client library though if you want to use them: https://github.com/nats-io/nats.ws. And yes, their servers "have websocket support".
Looks interesting. Coincidentally, I've just completed the bulk of work on a distributed Websocket network system to synchronize certain bits of state between multiple clients for my own kind of Storybook tool [0]. How interesting!
This kind of tool is exactly what I would have needed, instead of the approach I've taken which is a bit kludgy, grass-roots, novice-like, etc.
Good work :)
[0] https://github.com/samhuk/exhibitor/pull/22
MQTT-over-websocket does exist, and most MQTT brokers support it (Mosquito, AmazonMQ etc.). You're right about the compaction - MQTT doesn't have anything in it's protocol about compaction, and I don't know of any brokers that implement it. Having said that, you could use an MQTT-kafka bridge.
Something like Mosquito + https://github.com/nodefluent/mqtt-to-kafka-bridge + Redpanda in a docker image would work, though obv. this might be a bit overkill for most. Having said that, it does open many new avenues for interaction at scale. You pays your money...
Nats is not something I see as a competitor for external clients (browsers, mobile apps), primarily because it doesn't handle reconnections / message delivery / quality-of-service / at-least-once or exactly-once delivery (except for MQTT).
> When the connection is lost, your application would have to re-create it and all subscriptions if any. https://github.com/nats-io/stan.go#connection-status
Therefore, I don't see what it adds here. It seems designed for service communication, not client-server. They also don't list browsers as a use case https://docs.nats.io/nats-concepts/overview#use-cases. (though it is of course possible, it's just not ideal IMHO.)
They still have a js/browser client library though if you want to use them: https://github.com/nats-io/nats.ws. And yes, their servers "have websocket support".
https://github.com/centrifugal/centrifugo
It's a complete solution, including server, admin panel and client library.
We're an European company and use OVH, Hetzner and others.