ui-mock VS thirdroom

It's quite possible to develop Rust for Windows without using Windows.

Try my open source "ui-mock".[1] This is a test of the cross-platform stack. Just get the repository with "git clone", and make sure you have Rust installed for target "x86_64-pc-windows-gnu". See the Cargo.toml file for build instructions.

This is a game-type user interface. It's just some menus and a 3D cube. It doesn't do much, but it exercises all the lower levels. This allows debugging cross-platform problems in a simple environment. The main crates used are winit (cross-plaform window event handling), wgpu (cross-plaform GPU handling), rfd (cross-platform file dialogs), keychain (cross-platform password storage), egui (Rust-native menus and dialogs), and rend3 (safe interface to wgpu). For graphics, it uses Vulkan, so it will run on Windows back to the last release of Windows 7. Not Windows 95, though; it's 64-bit. It will also run under Wine, so you don't even need a Windows system to test.

My metaverse client uses the same stack. It's compiled on Linux, and runs on both Linux and Windows. So I'm building a high-performance 3D graphics program for Windows without even owning a Windows system or using any Microsoft software.

[1] https://github.com/John-Nagle/ui-mock

Profiling on the CPU side is well handled by tracy, which is a game-oriented profiler. My programs render-bench and ui-mock are prepped for Tracy, as is Rend3, so you can try it out on them.

ui-mock -- game GUI test fixture This exercises rfd->egui->rend3->wgpu. It's a game GUI with menus and dialogs, but no game behind it, just a 3D drawing of a cube. It's useful for making bugs in that stack repeatable. That's been helpful in wringing out obscure bugs in egui.

Egui works well with Rend3. Here's my example and library for that. It's a dummy game UI; no game, but brings up menus atop Rend3 3D. Egui is very low level. Each dialog takes a lot of code. Something to generate dialogs from some kind of template would be useful. I have many of those to do. Incidentally, does anyone have examples of good color themes for egui? The default is shades of black on black, which is a bit harsh. I'd like to see some examples where the aesthetics are better.

There's an attempt to make this work for Rust desktop applications. There's the winit crate, which does cross-platform windowing and event loops. There's egui, for menus and subwindows. There's rfd, for file dialogs, which are special for security reasons. And there's wgpu, for cross-platform 3D.

I'm using all of these in my ui-mock,[1] which is a GUI for a game without the game. It has 3D graphics with 2D GUI elements on top. I'm using this to shake down all the cross-platform problems for my metaverse client. My own code, which is 100% safe Rust, has no platform dependent code.

Results are pretty good. There's minor dirty laundry in those libraries, which has been reported to the various maintainers. Stuff like this:

- You can get a file dialog hidden behind the main window, which, in a full screen program, is a real problem. Mostly a Linux problem; works fine on Windows.

- Full screen on Windows mode under Wine 7 crashes Wine. Known Wine bug.

- Warnings from WGPU, but it works around all of them with some minor performance loss.

- Cross-platform packaging, to make a Windows installer without Windows, isn't implemented yet.

So, not big stuff. A lot of stuff works that you might not expect to work, such as profiling with tracy. Wgpu is taking care of Vulkan vs Apple's Metal. (Apple just had to Think Different, to the annoyance of everybody doing 3D.) Opening a web page in the default browser is cross-platform. You can cross-compile - I build the Windows version on Linux, without using any Microsoft tools.

With some more work, I could make this work on WASM and Android as well, but that requires some special casing, mostly because WASM doesn't have proper threads.

So cross-platform desktop development is working pretty well. Most of the problems I'm running into would not appear in a more typical application.

[1] https://github.com/John-Nagle/ui-mock

I have a Rend3/Egui/WGPU program, https://github.com/John-Nagle/ui-mock

Not sure if Mozilla Hubs federates, but as already said, ThirdRoom is Matrix VRchat thing.

you can calculate 2D CSS transforms which match the equivalent transforms of your WebGL scene in WebXR - as an efficient but hacky way to (for instance) do live video overlays in 3D without having to mess around importing the video texture into WebGL (assuming you don’t need occlusion or environmental effects etc).

we’re toying with this as an approach for video overlays in https://thirdroom.io, especially for underpowered devices.

by “matrix chat 3d like things” do you mean https://thirdroom.io?

It would be super easy to build something like this on Third Room - and then get e2ee and decentralisation etc for free :)

oh, wow. we have 3D Tile support in https://thirdroom.io but had only ever found NASA’s Mars dataset as a good set of tiles to point it at. This could effectively turn Third Room into a FOSS, decentralised, E2EE multiplayer Google Earth running over Matrix!

AT proto has some significant similarities to Matrix:

* Both are work by self-authenticating git-style replication of Merkle trees/DAGs

* Both define strict data schemas for extensible sets of events (Matrix uses JSON schema - https://github.com/matrix-org/matrix-spec/tree/main/data/eve... and OpenAPI; AT uses Lexicons)

* Both use HTTPS for client-server and server-server traffic by default.

* Both are focused on decentralised composable reputation - e.g. https://matrix.org/blog/2020/10/19/combating-abuse-in-matrix... on the Matrix side, or https://paulfrazee.medium.com/the-anti-parler-principles-for... on the bluesky side, etc.

* Both are designed as big-world communication networks. You don't have the server balkanisation that affects ActivityPub.

* Both eschew cryptocurrency systems and incentives.

There are some significant differences too:

* Matrix aspires to be the secure communication layer for the open web.

* AT aspires (i think) to be an open decentralised social networking protocol for the internet.

* AT has portable identity by default. We've been working on this on Matrix (e.g. MSC1228 - https://github.com/matrix-org/matrix-spec-proposals/pull/122... and MSC2787 - https://github.com/matrix-org/matrix-spec-proposals/blob/nei...) and have a new MSC (and implementation on Dendrite) in progress right now which combines the best bits of MSC1228 & MSC2787 into something concrete, at last. In fact the proto-MSC is due to emerge today.

* AT is proposing a asymmetrical federation architecture where user data is stored on Personal Data Servers (PDS), but indexing/fan-out/etc is done by Big Graph Servers (BGS). Matrix is symmetrical and by default federates full-mesh between all servers participating in a conversation, which on one hand is arguably better from a self-sovereignty and resilience perspective - but empirically has created headaches where an underpowered server joins some massive public chatroom and then melts. Matrix has improved this by steady optimisation of both protocol and implementation (i.e. adding lazy loading everywhere - e.g. https://matrix-org.github.io/synapse/latest/development/syna...), but formalising an asymmetrical architecture is an interesting different approach :)

* AT is (today) focused on for public conversations (e.g. prioritising big-world search and indexing etc), whereas Matrix focuses both on private and public communication - whether that's public chatrooms with 100K users over 10K servers, or private encrypted group conversations. For instance, one of Matrix's big novelties is decentralised access control without finality (https://matrix.org/blog/2020/06/16/matrix-decomposition-an-i...) in order to enforce access control for private conversations.

* Matrix also provides end-to-end encryption for private conversations by default, today via Double Ratchet (Olm/Megolm) and in the nearish future MLS (https://arewemlsyet.com). We're also starting to work on post quantum crypto.

* Matrix is obviously ~7 years older, and has many more use cases fleshed out - whether that's native VoIP/Video a la Element Call (https://element.io/blog/introducing-native-matrix-voip-with-...) or virtual worlds like Third Room (https://thirdroom.io) or shared whiteboarding (https://github.com/toger5/TheBoard) etc.

* AT's lexicon approach looks to be a more modular to extend the protocol than Matrix's extensible event schemas - in that AT lexicons include both RPC definitions as well as the schemas for the underlying datatypes, whereas in Matrix the OpenAPI evolves separately to the message schemas.

* AT uses IPLD; Matrix uses Canonical JSON (for now)

* Matrix is perhaps more sophisticated on auth, in that we're switching to OpenID Connect for all authentication (and so get things like passkeys and MFA for free): https://areweoidcyet.com

* Matrix has an open governance model with >50% of spec proposals coming from the wider community these days: https://spec.matrix.org/proposals

* AT has done a much better job of getting mainstream uptake so far, perhaps thanks to building a flagship app from day one (before even finishing or opening up the protocol) - whereas Element coming relatively late to the picture has meant that Element development has been constantly slowed by dealing with existing protocol considerations (and even then we've had constant complaints about Element being too influential in driving Matrix development).

* AT backs up all your personal data on your client (space allowing), to aid portability, whereas Matrix is typically thin-client.

* Architecturally, Matrix is increasingly experimenting with a hybrid P2P model (https://arewep2pyet.com) as our long-term solution - which effectively would end up with all your data being synced to your client. I'd assume bluesky is consciously avoiding P2P having been overextended on previous adventures with DAT/hypercore: https://github.com/beakerbrowser/beaker/blob/master/archive-.... Whereas we're playing the long game to slowly converge on P2P, even if that means building our own overlay networks etc: https://github.com/matrix-org/pinecone

I'm sure there are a bunch of other differences, but these are the ones which pop to the top of my head, plus I'm far from an expert in AT protocol.

It's worth noting that in the early days of bluesky, the Matrix team built out Cerulean (https://matrix.org/blog/2020/12/18/introducing-cerulean) as a demonstration to the bluesky team of how you could build big-world microblogging on top of Matrix, and that Matrix is not just for chat. We demoed it to Jack and Parag, but they opted to fund something entirely new in the form of AT proto. I'm guessing that the factors that went into this were: a) wanting to be able to optimise the architecture purely for social networking (although it's ironic that ATproto has ended up pretty generic too, similar to Matrix), b) wanting to be able to control the strategy and not have to follow Matrix's open governance model, c) wanting to create something new :)

From the Matrix side; we keep in touch with the bluesky team and wish them the best, and it's super depressing to see folks from ActivityPub and Nostr throwing their toys in this manner. It reminds me of the unpleasant behaviour we see from certain XMPP folks who resent the existence of Matrix (e.g. https://news.ycombinator.com/item?id=35874291). The reality is that the 'enemy' here, if anyone, are the centralised communication/social platforms - not other decentralisation projects. And even the centralised platforms have the option of seeing the light and becoming decentralised one day if we play our parts well.

What would be really cool, from my perspective, would be if Matrix ended up being able to help out with the private communication use cases for AT proto - as we obviously have a tonne of prior art now for efficient & audited E2EE private comms and decentralised access control. Moreover, I /think/ the lexicon approach in AT proto could let Matrix itself be expressed as an AT proto lexicon - providing interop with existing Matrix rooms (at least semantically), and supporting existing Matrix clients/SDKs, while using AT proto's ID model and storing data in PDSes etc. Coincidentally, this matches work we've been doing on the Matrix side as part of the MIMI IETF working group to figure out how to layer Matrix on top of other existing protocols: e.g. https://datatracker.ietf.org/doc/draft-ralston-mimi-matrix-t... and https://datatracker.ietf.org/doc/draft-ralston-mimi-matrix-m... - and if I had infinite time right now I'd certainly be trying to map Matrix's CS & SS APIs onto an AT proto lexicon to see what it looks like.

TL;DR: I think AT proto is cool, and I wish that open projects saw each other as fellow travellers rather than competitors.

You could take a look to this project: https://github.com/matrix-org/thirdroom As part of the project they are implementing a web Game engine, uses ThreeJS for graphics, and includes multiplayer using the Matrix protocol. Could be a good starting point.

Have you looked into what thirdroom or ambient have done in the space?

Have you lot made any friends over at Third Room yet? ;)

If you wanted to be serious about it, you could try to make something based on https://thirdroom.io