hocuspocus VS cr-sqlite

Hi HN! We're Nick, Patrick, Philip, Sebastian, Sven, and Timo from Titap (https://tiptap.dev/), an open source developer toolkit for building collaborative editing apps. Our editor framework, based on ProseMirror, is at https://github.com/ueberdosis/tiptap, and our real-time collaboration backend, based on Yjs, is at https://github.com/ueberdosis/hocuspocus.

Building editor interfaces like Notion or Google Docs in your web app takes a lot of work and time. Our open source tools and cloud services let you build collaborative content editing faster—in days or weeks, rather than months or years. And this is just for the editor. If you want real-time collaboration or other advanced features like version history in your editor, the overall workload quickly escalates—you will need a robust and serious backend infrastructure that requires even more time to set up and maintain. This doesn’t make sense for most frontend developers or most startups.

We spent eight years as a digital agency developing applications with complex content editing functionality. We learned the hard way how limited the existing editors were. After building Tiptap as a headless editor framework with an extension-based architecture, we needed to allow multiple users to edit content simultaneously, which got complicated. There was no simple solution that could be integrated quickly. So we built that too.

The Tiptap editor is based on the JS framework ProseMirror, which is a good foundation for editors. The learning curve for ProseMirror is steep because it's complicated to understand and lacks simple APIs and documentation. It takes a lot of code around ProseMirror to develop a modern user experience. We’ve taken care of that for you.

Tiptap is headless, so it will work with whatever frontend or design you have in mind—we make no assumptions about your UI. You can use it to develop block-based editors like Notion, classic interfaces like Google Docs, or whatever you need. It's also framework agnostic, so you can use it with React, Vue, etc., or vanilla JavaScript. And it's highly customizable through our extension architecture. We also provide an API to access ProseMirror's internals through Tiptap if you want to dig deep into the core.

Adding real-time collaboration to your editor is as easy as installing and configuring an extension. Our collaboration backend, called Hocuspocus, uses Yjs. This is a widely used implementation of CRDTs (conflict- free replicated data type). Hocuspocus makes it easy to set up a Node.js websocket server to handle communication between multiple peers to synchronize data. Like the Tiptap editor, Hocuspocus is designed to be extensible according to your needs. Also, Hocuspocus can work independently of Tiptap with other editors like Lexical or Slate.

An earlier version of Tiptap got discussed a couple years ago at https://news.ycombinator.com/item?id=26901975. We’ve been enjoying wider adoption since then. For example, Substack uses Tiptap for their editor that allows creators to write content on substack.com, and YC uses Tiptap in their Bookface forum (which is basically HN for YC alums).

With the Tiptap Cloud, we offer managed backend services if you don't want to build and maintain every feature yourself. For real-time collaboration, we provide a cloud infrastructure with multiple datacenter regions where you can deploy Hocuspocus. The Tiptap AI integration beta is a service where you connect your OpenAI API key to our backend and install the Tiptap editor AI extension to get AI writing experience in your editor. Here’s a demo: https://ai-demo.tiptap.dev/

We invite you to explore Tiptap's capabilities in your app, contribute to its open source development, and (hopefully!) join our welcoming community. We'd love to hear what you've already built with Tiptap or what's stopping you from creating something with it :-) We look forward to all of your comments!

Its Release Day: Tiptap Stable Release, Tiptap Collab & Hocuspocus 2.0.0 are out!

We set ourselves some bold goals this year and are delivering with our biggest releases yet.

Tiptap Editor Stable Release

We have released the 2.0.0 stable version of the Tiptap Editor. After 24 months in beta and a challenging last year, we have taken Tiptap to the next level! From now on, semantic versioning applies. The Tiptap editor is now an adult: https://github.com/ueberdosis/tiptap/releases/tag/v2.0.0

Tiptap Collab Release Beta

After six months of work, we finally announce the launch of Tiptap Collab, the Plug'n'Play real-time synchronization and collaboration cloud for your app. It's Hocuspocus on steroids in the cloud: https://tiptap.dev/hocuspocus/server/cloud

All you need is a Tiptap Pro account and a few lines of code in your Tiptap editor, and everything is in sync. No matter what content you've stored in your Tiptap editor.

And guess what? You can use Tiptap Collab with every app! Since Tiptap Collab is based on Yjs, you can use it for every application, as long as you use Yjs. It’s done with a few lines of code in your app.

Hocuspocus 2.0.0 Release

After the stable release in January 2023, Hocuspocus gets its first major version update with the new core feature called “Multiplexing”. Multiplexing allows you to synchronize multiple documents over the same websocket connection. This is especially useful when working with multiple documents at the same time and will make this experience even faster: https://github.com/ueberdosis/hocuspocus/releases/tag/v2.0.0

We are excited to invest even more time and commitment into Tiptap and especially look forward to your contribution!

Tiptap is a product baked by and for the community and will be even more this year.

I'm a happy user of https://github.com/vlcn-io/cr-sqlite/

If you're interested in this, here are some related projects that all take slightly different approaches:

- LiteSync directly competes with Marmot and supports DDL sync, but is closed source commercial (similar to SQLite EE): https://litesync.io

- dqlite is Canonical's distributed SQLite that depends on c-raft and kernel-level async I/O: https://dqlite.io

- cr-sqlite is a Rust-based loadable extension that adds CRDT changeset generation and reconciliation to SQLite: https://github.com/vlcn-io/cr-sqlite

Slightly related but not really (no multi writer, no C-level SQLite API or other restrictions):

- comdb2 (Bloombergs multi-homed RDMS using SQLite as the frontend)

- rqlite: RDMS with HTTP API and SQLite as the storage engine, used for replication and strong consistency (does not scale writes)

- litestream/LiteFS: disaster recovery replication

- liteserver: active read-only replication (predecessor of LiteSync)

Theory is great, but how can we apply this in practice? Instead of starting from 0, and writing a CRDT, let's try and leverage an existing project to do the heavy lifting. My choice is crSQLITE, an extension for SQLite to support CRDT merging of databases. Under the hood, the extension creates tables to track changes and allow inserting into an event log for merging states of separated peers.

Also https://github.com/vlcn-io/cr-sqlite/ which is SQLite + CRDTs

Runs/syncs to the browser too which is just lovely.

If you need multiple writers and can handle eventual correctness, you should really be using cr-sqlite[1]. It'll allow you to have any number of workers/clients that can write locally within the same process (so no network overhead) but still guarantee converge to the same state.

[1] https://github.com/vlcn-io/cr-sqlite

I am fully on the offline-first bandwagon after starting to use cr-sqlite (https://vlcn.io), which works similar to ElectricSQL.

I thought the bundle size of wasm-sqlite would be prohibitive, but it's surprisingly quick to download and boot. Reducing network reliance solves so many problems and corner-cases in my web app. Having access to local data makes everything very snappy too - the user experience is much better. Even if the user's offline data is wiped by the browser (offline storage limits are a bit of a minefield), it is straightforward to get all synced changes back from the server.

I didn't know that. Especially the first approach sounds interesting to me, because as far as I know the transactions of Yjs seem to be a problem on heavily changing documents. https://github.com/vlcn-io/cr-sqlite#approach-1-history-free... Thanks!

I wonder how this compares to https://vlcn.io?

The short ask: Anyone know of any projects that bring incremental view maintenance to SQLite?

The why:

Applications are usually read heavy. It is a sad state of affairs that, for these kinds of apps, we don't put more work on the write path to allow reads to benefit.

Would the whole No-SQL movement ever even have been a thing if relational databases had great support for materialized views that updated incrementally? I'd like to think not.

And more context:

I'm working to push the state of "functional relational programming" [1], [2] further forward. Materialized views with incremental updates are key to this. Bringing them to SQLite so they can be leveraged one the frontend would solve this whole quagmire of "state management libraries." I've been solving the data-sync problem in SQLite (https://vlcn.io/) and this piece is one of the next logical steps.

If nobody knows of an existing solution, would love to collaborate with someone on creating it.

[1] - https://github.com/papers-we-love/papers-we-love/blob/main/design/out-of-the-tar-pit.pdf