thirdroom VS A-Frame

Not sure if Mozilla Hubs federates, but as already said, ThirdRoom is Matrix VRchat thing.

you can calculate 2D CSS transforms which match the equivalent transforms of your WebGL scene in WebXR - as an efficient but hacky way to (for instance) do live video overlays in 3D without having to mess around importing the video texture into WebGL (assuming you don’t need occlusion or environmental effects etc).

we’re toying with this as an approach for video overlays in https://thirdroom.io, especially for underpowered devices.

by “matrix chat 3d like things” do you mean https://thirdroom.io?

It would be super easy to build something like this on Third Room - and then get e2ee and decentralisation etc for free :)

oh, wow. we have 3D Tile support in https://thirdroom.io but had only ever found NASA’s Mars dataset as a good set of tiles to point it at. This could effectively turn Third Room into a FOSS, decentralised, E2EE multiplayer Google Earth running over Matrix!

AT proto has some significant similarities to Matrix:

* Both are work by self-authenticating git-style replication of Merkle trees/DAGs

* Both define strict data schemas for extensible sets of events (Matrix uses JSON schema - https://github.com/matrix-org/matrix-spec/tree/main/data/eve... and OpenAPI; AT uses Lexicons)

* Both use HTTPS for client-server and server-server traffic by default.

* Both are focused on decentralised composable reputation - e.g. https://matrix.org/blog/2020/10/19/combating-abuse-in-matrix... on the Matrix side, or https://paulfrazee.medium.com/the-anti-parler-principles-for... on the bluesky side, etc.

* Both are designed as big-world communication networks. You don't have the server balkanisation that affects ActivityPub.

* Both eschew cryptocurrency systems and incentives.

There are some significant differences too:

* Matrix aspires to be the secure communication layer for the open web.

* AT aspires (i think) to be an open decentralised social networking protocol for the internet.

* AT has portable identity by default. We've been working on this on Matrix (e.g. MSC1228 - https://github.com/matrix-org/matrix-spec-proposals/pull/122... and MSC2787 - https://github.com/matrix-org/matrix-spec-proposals/blob/nei...) and have a new MSC (and implementation on Dendrite) in progress right now which combines the best bits of MSC1228 & MSC2787 into something concrete, at last. In fact the proto-MSC is due to emerge today.

* AT is proposing a asymmetrical federation architecture where user data is stored on Personal Data Servers (PDS), but indexing/fan-out/etc is done by Big Graph Servers (BGS). Matrix is symmetrical and by default federates full-mesh between all servers participating in a conversation, which on one hand is arguably better from a self-sovereignty and resilience perspective - but empirically has created headaches where an underpowered server joins some massive public chatroom and then melts. Matrix has improved this by steady optimisation of both protocol and implementation (i.e. adding lazy loading everywhere - e.g. https://matrix-org.github.io/synapse/latest/development/syna...), but formalising an asymmetrical architecture is an interesting different approach :)

* AT is (today) focused on for public conversations (e.g. prioritising big-world search and indexing etc), whereas Matrix focuses both on private and public communication - whether that's public chatrooms with 100K users over 10K servers, or private encrypted group conversations. For instance, one of Matrix's big novelties is decentralised access control without finality (https://matrix.org/blog/2020/06/16/matrix-decomposition-an-i...) in order to enforce access control for private conversations.

* Matrix also provides end-to-end encryption for private conversations by default, today via Double Ratchet (Olm/Megolm) and in the nearish future MLS (https://arewemlsyet.com). We're also starting to work on post quantum crypto.

* Matrix is obviously ~7 years older, and has many more use cases fleshed out - whether that's native VoIP/Video a la Element Call (https://element.io/blog/introducing-native-matrix-voip-with-...) or virtual worlds like Third Room (https://thirdroom.io) or shared whiteboarding (https://github.com/toger5/TheBoard) etc.

* AT's lexicon approach looks to be a more modular to extend the protocol than Matrix's extensible event schemas - in that AT lexicons include both RPC definitions as well as the schemas for the underlying datatypes, whereas in Matrix the OpenAPI evolves separately to the message schemas.

* AT uses IPLD; Matrix uses Canonical JSON (for now)

* Matrix is perhaps more sophisticated on auth, in that we're switching to OpenID Connect for all authentication (and so get things like passkeys and MFA for free): https://areweoidcyet.com

* Matrix has an open governance model with >50% of spec proposals coming from the wider community these days: https://spec.matrix.org/proposals

* AT has done a much better job of getting mainstream uptake so far, perhaps thanks to building a flagship app from day one (before even finishing or opening up the protocol) - whereas Element coming relatively late to the picture has meant that Element development has been constantly slowed by dealing with existing protocol considerations (and even then we've had constant complaints about Element being too influential in driving Matrix development).

* AT backs up all your personal data on your client (space allowing), to aid portability, whereas Matrix is typically thin-client.

* Architecturally, Matrix is increasingly experimenting with a hybrid P2P model (https://arewep2pyet.com) as our long-term solution - which effectively would end up with all your data being synced to your client. I'd assume bluesky is consciously avoiding P2P having been overextended on previous adventures with DAT/hypercore: https://github.com/beakerbrowser/beaker/blob/master/archive-.... Whereas we're playing the long game to slowly converge on P2P, even if that means building our own overlay networks etc: https://github.com/matrix-org/pinecone

I'm sure there are a bunch of other differences, but these are the ones which pop to the top of my head, plus I'm far from an expert in AT protocol.

It's worth noting that in the early days of bluesky, the Matrix team built out Cerulean (https://matrix.org/blog/2020/12/18/introducing-cerulean) as a demonstration to the bluesky team of how you could build big-world microblogging on top of Matrix, and that Matrix is not just for chat. We demoed it to Jack and Parag, but they opted to fund something entirely new in the form of AT proto. I'm guessing that the factors that went into this were: a) wanting to be able to optimise the architecture purely for social networking (although it's ironic that ATproto has ended up pretty generic too, similar to Matrix), b) wanting to be able to control the strategy and not have to follow Matrix's open governance model, c) wanting to create something new :)

From the Matrix side; we keep in touch with the bluesky team and wish them the best, and it's super depressing to see folks from ActivityPub and Nostr throwing their toys in this manner. It reminds me of the unpleasant behaviour we see from certain XMPP folks who resent the existence of Matrix (e.g. https://news.ycombinator.com/item?id=35874291). The reality is that the 'enemy' here, if anyone, are the centralised communication/social platforms - not other decentralisation projects. And even the centralised platforms have the option of seeing the light and becoming decentralised one day if we play our parts well.

What would be really cool, from my perspective, would be if Matrix ended up being able to help out with the private communication use cases for AT proto - as we obviously have a tonne of prior art now for efficient & audited E2EE private comms and decentralised access control. Moreover, I /think/ the lexicon approach in AT proto could let Matrix itself be expressed as an AT proto lexicon - providing interop with existing Matrix rooms (at least semantically), and supporting existing Matrix clients/SDKs, while using AT proto's ID model and storing data in PDSes etc. Coincidentally, this matches work we've been doing on the Matrix side as part of the MIMI IETF working group to figure out how to layer Matrix on top of other existing protocols: e.g. https://datatracker.ietf.org/doc/draft-ralston-mimi-matrix-t... and https://datatracker.ietf.org/doc/draft-ralston-mimi-matrix-m... - and if I had infinite time right now I'd certainly be trying to map Matrix's CS & SS APIs onto an AT proto lexicon to see what it looks like.

TL;DR: I think AT proto is cool, and I wish that open projects saw each other as fellow travellers rather than competitors.

You could take a look to this project: https://github.com/matrix-org/thirdroom As part of the project they are implementing a web Game engine, uses ThreeJS for graphics, and includes multiplayer using the Matrix protocol. Could be a good starting point.

Have you looked into what thirdroom or ambient have done in the space?

Have you lot made any friends over at Third Room yet? ;)

If you wanted to be serious about it, you could try to make something based on https://thirdroom.io

I looked at it seriously for content authoring but gave it up.

The big problem is you cannot import images, textures, 3-d models and such from ordinary tools. You have something like constructive solid geometry to work with but only so much and there is a slider you can use to set the number of players and the more players the less geometry you can use.

I want to make worlds based in photographs (particularly pano and stereo) and art. McDonalds needs to put a Coca-Cola logo on the side of the cup. Either way it is a non-starter.

HW supports collaboration (more than one person shares the world) but https://aframe.io/ lets me make the content I want. If I have to choose one or the other I am going to pick the second.

My take on Meta Quest is that it seems highly successful as a gaming environment based on an app store but is skews towards single-player experiences. Like a lot of AAA games, the excellent Asgard’s Wrath 2 has some multiplayer tacked on but it is all meaningless like leaderboards and the occasional ghost that shows up in a procedurally generated dungeon.

Of course, Meta wants to make multiplayer experiences but somehow they just can’t do it.

... I remember inventing signals circa 2005 for a knowledge graph editor I worked on in GWT back when I didn't care about knowledge graphs. Not long after that I was using similar concepts to develop a complex Silverlight decision support application for sales territories.

I'm still a little appalled that people have forgotten how to make HTML form applications like we made in 1999 and frequently use React for things that don't really need it. React has, I think, one of the most elegant methods of implementing a widget set ever invented (sure beats XAML, FXML, Visual Basic, etc.) but it doesn't support a real answer to maintaining application state that would be adequate for applications as complex as RIAs made by early adopters in 2005.

My favorite use for React is

https://aframe.io/

which works like a cross between VRML and Unity Framework. Here you can easily build a virtual world that people can experience with desktop, mobile and VR headsets. Something you can't do with HTML forms!

I have a few side projects based on HTMX but I am pretty sure we will see something like it but better soon. In particular back end frameworks need to be updated for the new reality. Often you have a "component" like a dropdown box which can either be drawn as part of a bigger page or updated on its own and you really need some way you can reuse the HTML template for those purposes and hopefully also reuse associated code that does database lookups and such.

A lot depends on what kind of app you want to build.

For my current side projects I am interested in two kinds of technology.

(1) ML-enhanced apps that involve sorting and classifying content. Here I am drawing everything in the back end and using HTMX to partially update the screen to, for instance, create modal dialog boxes. The point here is that I want to quickly add new UI tasks and it will not be so quick if a front end build process is involved. (Notably in my system you can roll out a new task on the back end without reloading the front end)

My back end is Python which I am souring on. The basic problem is that it does not have threads. So if I have a 1GB ML model and a celery with 32 processes I could waste memory with 32 copies of the model. There are ways of working around all this but if I was just using Java, one copy of the model could be shared by any number of threads. Java doesn’t have the ML libraries though.

The general principle here is that the vast majority of React apps are just ordinary form processing apps that could be developed using the methods we used back in 1999 and seem to have either forgot or never learned (if we are young.). You could make the case that DX and UX were better then.

(2) Virtual reality applications using

https://aframe.io/

here there is no back end. With these tools you can make a world that is browsable with an ordinary web browser but put on a VR headset and you are there. This is another counter to the boring React app because it is something you could never do with an frontendless site.

Meta Quest also has full support for WebXR which lets you bypass all the app store BS. You can use this framework to make worlds visible on most XR headsets as well as desktop and mobile

https://aframe.io/

My understanding is that Apple is making excuses about being slow to support WebXR.

Meta Quest. It even runs on Android. For that matter, so does Magic Leap 2. If they deign to do so all the "usual suspects" who make Android phones could make Android-based VR headsets.

As much as the media has been obsessed with the train wreck that is Horizon Worlds, it's a well kept secret that Meta Quest has an app store that works like the app store on a game console. You can even sideload phone, tablet and TV apps and they "just work" most of the time.

There is no working "metaverse" and even meaningful multiplayer games are thin on the ground, but no shortage of good single-player games and what I'd call game-adjacent apps.

It's little recognized that XR apps are highly portable because they are almost always based on portable frameworks like Unity. In fact, just about every XR headset supports WebXR which makes it outright easy to make web-based virtual worlds

https://aframe.io/

these work with desktop, phones and tablets as well as most of the AR and VR headsets. All it takes is that you "think different" and choose to live life outside the app store.

WebXR is pretty amazing. You can make a site with

https://aframe.io/

which can be navigated with a browser on desktop and mobile. Using a VR headset you can give permission and… you’re in the world! It’s like one of those kids TV shows where people can jump into the world of a book.

With WebXR you can make applications that run on Meta Quest, Magic Leap 2, Hololens 2, PCVR and other platforms. It kinda drives me up the wall that so many people are hung up on Apple because AVP competes with quite a few different platforms that are all pretty similar and if this kind of thing catches on you’d better believe people will be porting applications between them…

To Facebook's credit, Facebook really is investing in developing a new platform, which is more than you can say about Google, Apple, Amazon, etc.

The reality is that Horizon Worlds is a joke (maximum 20 players in a world and you trade players for geometry, a 20 player world is much smaller than an 8 player world.) Horizon Worlds doesn't let you import images, video or sound.

A VR world has to fit everything you can see in a headset. A really big game like Asgard's Wrath 2 (absolutely amazing) is like an open world game which is full of hidden loading screens and it's quite expensive to develop if you want it to feel open.

By not letting you load those kind of resources HW keeps the size of the world under control, plus it disappears pornography, copyright violations and a whole lot of trouble. It kills HW for me because I want to make content based on photography and visual art, and kills it for commercial users because McDonald's just has to put a Coca-Cola logo on the side of the cup.

The MQ3 is basically an Android phone you wear on your face and the mainstream way to deliver apps for it is the same as a mobile phone. Unlike Apple, you can sideload, in fact you can sideload many ordinary phone apps, such as the Tailscale client or the Denon HEOS client and they usually work. It's not less any open an environment than mobile.

On top of that you can create webpages with WebXR that appear in VR at the touch of the button and this framework

https://aframe.io/

you can make worlds that are browsable on phone, tablet and desktop but that you can enter with a VR headset. (Makes me think of a few kids TV shows where the characters can jump into books) These are compatible across a range of devices such as the Hololens 2, PCVR headsets, and even the Apple Vision Pro.

Google and Microsoft provide plenty of models for half-baked efforts that end in failure (every time a B2B startup gets bought by Google the competitors have a meeting of their salespeople the next day and say... "YOU'RE ON COMMISSION, YOU'RE GOING TO GET RICH!) Facebook is the one big tech company that is seriously investing in a new platform and they deserve some credit for it, whereas all the rest of them are so wrapped up in "business theater" and "technology theater" that nobody realizes Google hasn't made anything since they bought YouTube.

Traditionally, VR and AR development involved complex frameworks and languages, but Node.js brings a fresh approach to the table. Its familiarity with JavaScript allows developers to write code for VR and AR applications with ease. By utilizing libraries like A-Frame for VR and AR.js for AR, developers can create immersive experiences directly in the browser using Node.js. This seamless integration opens up opportunities for a wider range of developers to enter the VR and AR landscape.

if there is none, how could it work? I'm not a dev, i work in design, UX, art. however this seems excellent system https://aframe.io yet I'm sure are others. runs in the browser, super fast, the code is just simple human readable, text, so works with HTML, JavaScript.