vrite VS marked

These reasons (and many others) are why I decided to create Vrite - an open-source developer content platform.

Vrite is an open-source developer content platform, featuring extensible editing experience, content management tools, and powerful APIs. It’s intended as an all-in-one, collaborative solution for product documentation, technical blogs, and knowledge bases.

So, I've been building Vrite as an open-source project for a while now, and I'm happy to finally share it here - with v0.2.0 now having official self-hosting support.

Let’s start by getting into Vrite. You can use the hosted version (free while Vrite is in Beta) or self-host Vrite from the source code (with better self-hosting support coming soon)

You can use Vrite via the hosted version (that’s free while in Beta) or self-host it from the open-source repo (though good support for self-hosting is still in the works).

Since Vrite (and Vrite Editor for that matter) is currently in Public Beta, new features and improvements are in active development. The best way to try it out right now is through the hosted version at app.vrite.io (free while in Beta) with better self-hosting support in the works.

The editor itself is a standalone app, extracted from the larger Vrite CMS project (https://github.com/vriteio/vrite) which you can also test out (only with sign-in) here: https://app.vrite.io/

The main output is JSON ProseMirror format. Other formats are processed from this JSON using Transformers and Vrite SDK: https://github.com/vriteio/vrite/tree/main/packages/sdk/java...

In the GFM transformer I try to follow GitHub Flavored Markdown spec, which technically doesn't support embeds. Since I didn't find any "common" syntax to use for the embeds, I just left them out. They're still there in JSON and HTML outputs.

That's one of the drawbacks of MD. That said, I plan to add an option like Markdoc, which has clearly defined spec for implementing custom blocks like embeds.

That said, for now, if you sign up for the full Vrite CMS, you can create a custom Transformer and process the output so that embeds are included in your desired format. That's what I'm doing for auto-publishing extensions for platforms like Dev.to and Hashnode. I don't know what your use-case is, but I thought it's worth noting.

The process basically came down to figuring out the position and size of the block node, given a selection of an entire top-level node or just its child node (source code):

Next, install gray-matter to extract metadata from the front matter of markdown files, and marked to convert the markdown files to HTML:

📑 Marked Markdown parser. Use it to create your own markdown editor.

Vrite SDK provides a few built-in input and output transformers. These are functions, with standardized signatures to process the content from and into Vrite. In this case, gfmInputTransformer is essentially a GitHub Flavored Markdown parser, using Marked.js under the hood.

Another contestant in this realm is Bright[1]. It runs entirely on the server and doesn't increase bundle size as seen here[2]. Regarding parsing speed tree-sitter is without a doubt performant since it is written in Rust, but I don't have any problems "parsing on every keystroke" with a setup containing Marked[3], highlight.js[4] and a sanitizer. I did however experience performance issues with other Markdown parser libraries than Marked.

[1]: https://bright.codehike.org/

[2]: https://aihelperbot.com/test-suite

[3]: https://github.com/markedjs/marked

[4]: https://highlightjs.org/

The project uses markedJS to convert markdown into HTML, this is their GitHub page.

To handle pasting block Markdown content like this, I had to tap into ProseMirror and implement a custom mechanism (though somewhat based on TipTap’s paste rules), detecting starting and ending points of the blocks and parsing them with Marked.js.

I am using marked for markdown parsing together with marked-highlighting to handle syntax highlighting and everything is working as it should.

I'm trying to render user input using SvelteMarkdown (that uses marked).

GPT API returns data in markdown format. You can parse it using a Markdown library and string manipulation. On Electron app I developed https://jhappsproducts.gumroad.com/l/gpteverywhere, I used https://github.com/markedjs/marked and a code syntax highlighting package to display code blocks. And used JavaScript string manipulation to detect when code blocks start and end so I could add COPY/SAVE buttons to the blocks. I hope this helps, and happy coding! :)

Again, with streaming enabled, you’ll now receive new tokens as soon as they’re available. Given that OpenAI’s API uses Markdown in its response format, a full message will need to be put together from the incoming tokens and parsed to HTML, as accepted by the replaceContent function. For this purpose, I’ve used the Marked.js parser.