unified
peritext
Our great sponsors
unified | peritext | |
---|---|---|
14 | 20 | |
4,244 | 615 | |
1.5% | 2.6% | |
7.2 | 0.0 | |
3 days ago | over 1 year ago | |
JavaScript | TypeScript | |
MIT License | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
unified
-
No CMS? Writing Our Blog in React
From TFA:
> My idea was that surely it's possible to write a bunch of markdown, and then have that get wrapped in a bunch of JSX tags that come pre-styled, using the styles of your existing repo? For example, what I expected was to be able to write *test* (Markdown for bold) and then get a component that looked liketest where is a library-defined React component
It surely is possible, so perhaps I can share some links if others are wondering the same thing.
If you like to roll your own solution for that, you can use the unified ecosystem: https://unifiedjs.com/
However, if you want JSX just do what everyone does and reach for MDX:
-
The hustle free way to build a next.js blog with dev.to
Styling the blog, figure out the eco-system within UnifiedJs, remark-rehype, oh boy, I could write another blog with that.
-
building a basic markdown editor: unified, trees and data
To build the Markdown editor (and the preview, mostly), I decided to use unified, an ecosystem of tools allowing the developer to parse a format into an abstract tree and back into another format (for example, markdown to html) and modify said tree (for example, to add specific classes to certain html elements before they are converted to an actual html string. The basics of how to do so can be found in this article, but they mostly consist of:
- Content as structured data, Compile content to syntax trees and vice versa
-
HTML to React service
It’ll take you a few hours to become proficient in the “unified” syntax tree libraries. So worth it. Easy fully customized conversion to/from html, react, plain text, markdown, you name it. https://unifiedjs.com/
-
Universal compiler using WASM architecture
Why universal? Because a lot of languages are simillar in a bunch of ways and it might be good to reuse existing parts of the compiler to speed up the process of writing new DSLs for example. Also a pretty big use-case is not for compilers itself but just for transformers like Markdown to HTML, there are already a lot of libraries (like https://github.com/unifiedjs/unified for ASTs and https://tree-sitter.github.io/tree-sitter/ for parsing) that try achieve similar goals but they are not working together.
-
Converting and customizing Markdown files to HTML with Unified, remark & rehype
Unified is a framework to process Markdown. It's a plugin-based tool that allows you to inspect and modify the way Markdown is converted to other formats like HTML.
-
Wrighter (β) - A Powerful Markdown Blogger & A Writing Companion ⚡
under the hood, the markdown is parsed by the unified remark and rehype processors, which in turn under the hood manipulate the markdown + HTML as an AST, which gives a lot of flexibility on parsing and rendering markdown. The editor uses them as plugins, which allows me to pick the features and inject them into the editor, one such injectable feature is the custom-made "copy from anywhere & paste as markdown" feature.
-
Building React Components from headless CMS markdown
Having the flexibility to create custom React Components for markdown is essential and with react-markdown, we can achieve this in just a few lines of code. Boosting performance, increasing link security, and having overall increased capabilities for our markdown data are what we achieve here. Aside from the improvements within our application, this component is super easy to use and taps us into the unifiedjs (giving us superpowers!).
-
I create my own homepage!
The Markdown processor used unified assets.
peritext
-
Cola: A text CRDT for real-time collaborative editing
This doesn’t appear to support rich text formatting ranges like bold, italic, etc - unless I’m missing something in the API. AFAIK Peritext is still the state of the art in rich text CRDT algorithms https://www.inkandswitch.com/peritext/
I’d love to see this build the rich text stuff from the Peritext algorithm.
-
The Cloud Is a Prison. Can the Local-First Software Movement Set Us Free?
The work Ink & Switch (unaffiliated) do has been an inspiration to my with regard to local-first and decentralized software: https://www.inkandswitch.com
They have a quasi-manifesto on local-first (https://www.inkandswitch.com/local-first/) and have published the best rich text CRDT around, Peritext: https://www.inkandswitch.com/peritext/
Lots of interesting work happening in this space.
-
Figma Is a File Editor
Take a look at https://automerge.org/ and the stack those folks are building. You're exactly right that it's a difficult balance (specifically the trick is proving commutativity for the domain-specific data of your application). But automerge (and then https://github.com/inkandswitch/peritext) show it's at least possible. Good stuff.
-
Ask HN: What is new in Algorithms / Data Structures these days?
Yes - The BFT problem only matters when you have Byzantine actors. But I think users deserve and expect the system to be reasonably well behaved and predictable in all situations. Anything publically writable, for example, needs BFT resilience. Or any video game.
As for the prosemirror problem, I assume you’re talking about weird merges from users putting markdown in a text crdt? You’re totally right - this is a problem. Text CRDTs treat documents as a simple sequence of characters. And that confuses a lot of structured formats. For example, if two users concurrently bold the same word, the system should see that users agree that it should be bolded. But if that “bold” intent is translated into “insert double asterisks here and here”, you end up with 4 asterisks before and after the text, and that confused markdown parsers. The problem is that a text crdt doesn’t understand markdown.
JSON editing has similar problems. I’ve heard of plenty of people over the years putting json text into a text crdt, only to find that when concurrent edits happen, the json grows parse errors. Eg if two users concurrently insert “a” and “b” into an empty list. The result is [“a””b”] which can’t be parsed.
The answer to both of these problems is to use CRDTs which understand the shape of your data structure. Eg, use a json OT/crdt system for json data (like sharedb or automerge). Likewise, if the user is editing rich text in prosemirror then you want a rich text crdt like peritext. Rich text CRDTs add the concept of annotations - so if two users bold overlapping regions of text, the crdt understands that the result should be that the entire region is bolded. And that can be translated back to markdown if you want.
The ink & switch people did a great write up of how this sort of crdt works here: https://www.inkandswitch.com/peritext/
- Edge cases in collaborative rich text editing (2021)
-
You might not need a CRDT
> I'm looking out for practical CRDT ideas that works well with richtext.
Have you seen Peritext from Ink & Switch? https://www.inkandswitch.com/peritext/ It's relatively new, but is a CRDT aimed at rich text!
-
CRDTs make multiplayer text editing part of Zed's DNA
To put it in a different perspective, plain text editing has well-solved CRDT patterns. But, semantic data-structures like rich-text or syntax trees is what's tricky and has unsolved challenges.
Peritext[1] is the only one that came close to solving rich-text, but even that one left out important aspect of rich-text editing like handling list & table operations as "work to be done later".
For people interested on why it's difficult to build CRDTs for richtext, here's a piece I wrote a year back: https://writer.zohopublic.com/writer/published/grcwy5c699d67...
Related HN discussion: https://news.ycombinator.com/item?id=29433896
[1] https://github.com/inkandswitch/peritext
- Peritext – A CRDT for Rich-Text Collaboration
-
Evan Wallace CRDT Algorithms
Anyone unsure of what a CRDT is, this is the perfect intro: https://www.inkandswitch.com/peritext/
The two most widely used CRDT implementations (combining JSON like general purpose types and rich text editing types) are:
- Automerge https://github.com/automerge/automerge
- Yjs https://github.com/yjs/yjs
-
Is Svelte capable of a Google Docs & Sheets clone?
Svelte is, but that is your smallest problem. You want to look into CRDTs (conflict-free replicated data types) to offer true (offline) collaboration. A popular JS library to solve this complex problem is called [automerge](Conflict-free replicated data type). A rather recent development in that area specifically for text-based content is Peritext. Also check out this interactive tutorial about CRDTs.
What are some alternatives?
mdx - Markdown for the component era
automerge - A JSON-like data structure (a CRDT) that can be modified concurrently by different users, and merged again automatically.
uniorg - An accurate Org-mode parser for JavaScript/TypeScript
y-crdt - Rust port of Yjs
orgajs - parse org-mode content into AST
dokieli - :bulb: dokieli is a clientside editor for decentralised article publishing, annotations and social interactions
retext - natural language processor powered by plugins part of the @unifiedjs collective
threlte - 3D framework for Svelte
ntast - Notion Abstract Syntax Tree specification.
automerge-rs - Rust implementation of automerge [Moved to: https://github.com/automerge/automerge]
unist - Universal Syntax Tree used by @unifiedjs
yjs - Shared data types for building collaborative software