webtransport VS turbo

WebTransport is now enabled by default and will be going to release with 114. As the original Explainer notes, it enables multiple use-cases that are hard or impossible to handle without it, especially for Gaming and live streaming. It covers cases that are problematic for alternative mechanisms, such as WebSockets. Built on top of HTTP3 (HTTP2 support will be coming later). The current implementation in Firefox is passing 505 out of 565 Web-Platform Tests.

Sort of. The browser will re-use the connection if you have a bunch of resources in the HTML. When rendering it sees that it needs 2 images and 3 javascript files from the same server, so it pipelines all of those. But for requests initiated from javascript, you're going to get a new connection for each one unless you're using a library that implements the long-polling hack. SocketIO can use the long-polling hack as a fallback if websockets is not supported. HTTP/2 (formerly SPDY) gets part of the way to replacing websockets, but it's not a synchronous link. Only the client can send messages to the server and the server can only respond to those message (with websockets, either side can send messages once the connection is open). FWIW, less than 50% of websites use HTTP/2. HTTP/3's webtransport looks like it could replace websockets, but it also looks like it'll live along side websockets.

If it's streaming data like dashboard statistics the new WebTransport API might be a much better base: https://github.com/w3c/webtransport/blob/main/explainer.md

Are you guys looking into the Web Transport protocol for the future? Right now you have to tunnel the websocket connections over http2 and it will probably be the same for http3 afaik.

I know there is this work in progress (https://w3c.github.io/webtransport/) and websockets are probably fine for a long time but sooner or later (unless there is an update to websockets) it will probably be faster to just do normal http requests and listen on server sent events.

What are your thoughts for Liveview for the future? Will it forever stay on websockets or would you be open to change the underlying technology if / when new stuff becomes available?

It's taken nearly 10 years for QUIC to be refined and adopted in the wild and we're basically there. There's even a new browser API in the works called WebTransport.

tl;dr using WebRTC just for realtime client<->server data sucks, but WebTransport[1] is coming soon to serve that exact usecase with an easy API

WebRTC has data channels, which are currently the only way to achieve unreliable and unordered real-time communication (UDP-style) between the browser and other browsers or a server. This is pretty essential for any networked application where latency is critical, like voice and video and fast-paced multiplayer games.

As other commenters have noted, it's a royal pain in the ass to set up WebRTC if all you want is UDP-style communication between a server and browser, since you need to wrangle half a dozen other protocols in the process.

However! A new API, WebTransport[1], is actively being developed that will offer a WebSockets-like (read: super simple to set up) API for UDP-style communication. I am extremely excited about it and its potential for real-time browser-based multiplayer games (which I'm working on).

https://github.com/w3c/webtransport

Supermemory has three main modules, managed by turborepo:

From a structural viewpoint, we use a monorepo (Turborepo) to manage packages, even though we currently have only one Next.js app. We chose this setup because it prepares us for future developments, which will include additional apps. This arrangement helps keep the packages well-separated and self-contained.

# src Dockerfile: https://github.com/vercel/turbo/blob/main/examples/with-docker/apps/web/Dockerfile FROM node:18-alpine AS alpine # setup pnpm on the alpine base FROM alpine as base ENV PNPM_HOME="/pnpm" ENV PATH="$PNPM_HOME:$PATH" RUN corepack enable RUN pnpm install turbo --global FROM base AS builder # Check https://github.com/nodejs/docker-node/tree/b4117f9333da4138b03a546ec926ef50a31506c3#nodealpine to understand why libc6-compat might be needed. RUN apk add --no-cache libc6-compat RUN apk update # Set working directory WORKDIR /app COPY . . RUN turbo prune --scope=web --docker # Add lockfile and package.json's of isolated subworkspace FROM base AS installer RUN apk add --no-cache libc6-compat RUN apk update WORKDIR /app # First install the dependencies (as they change less often) COPY .gitignore .gitignore COPY --from=builder /app/out/json/ . COPY --from=builder /app/out/pnpm-lock.yaml ./pnpm-lock.yaml COPY --from=builder /app/out/pnpm-workspace.yaml ./pnpm-workspace.yaml RUN pnpm install # Build the project COPY --from=builder /app/out/full/ . COPY turbo.json turbo.json RUN turbo run build --filter=web # use alpine as the thinest image FROM alpine AS runner WORKDIR /app # Don't run production as root RUN addgroup --system --gid 1001 nodejs RUN adduser --system --uid 1001 nextjs USER nextjs COPY --from=installer /app/apps/web/next.config.js . COPY --from=installer /app/apps/web/package.json . # Automatically leverage output traces to reduce image size # https://nextjs.org/docs/advanced-features/output-file-tracing COPY --from=installer --chown=nextjs:nodejs /app/apps/web/.next/standalone ./ COPY --from=installer --chown=nextjs:nodejs /app/apps/web/.next/static ./apps/web/.next/static COPY --from=installer --chown=nextjs:nodejs /app/apps/web/public ./apps/web/public CMD node apps/web/server.js

Following this example: https://github.com/vercel/turbo/tree/main/examples/with-docker/apps/web. Except I'm using pnpm. Edit Reddit Codeblocks are horrible and keeps removing all formatting.

The Dockerfile might seem a bit complicated (it is), but the reason for that is mostly just turborepo and the need for good caching. Realistically, you will only need to change the last line, if at all. It is based on this awesome Github Issue.

PURISTA is organized in a monorepo. During the development and build process, Turbo is used to execute different tasks and steps on multiple packages with one command.

Turbo repo

Previously we mapped each Vercel project to a single Supabase project. With this release, we're introducing the concept of project 'Connections'. Supabase projects can have an unlimited number of Vercel Connections. This is especially useful for monorepos using Turborepo.

One detail I enjoy from this post is that sometimes you can just call a CLI[0]. It's easy to spend a lot of time figuring out how to expose some Rust/C code as a library for your language, but I like the simplicity of just compiling, shipping the binary and then calling it as a subprocess.

Yes, there's overhead in starting a new process to "just call a function", but I think this approach is still underutilized.

[0]: https://github.com/vercel/turbo/blob/c0ee0dea7388d1081512c93...