esprima VS ProseMirror

> check out the Web X-Ray repo <https://github.com/mozilla/goggles.mozilla.org/>.

Thanks for example. Peeking a bit under the hood, it appears to be due to transitive dependencies referencing github urls (and transient ones at that) instead of semver, which admittedly is neither standard nor good practice...

FWIW, simply removing `"grunt-contrib-jshint": "~0.4.3",` from package.json and related jshint-related code from Gruntfile was sufficient to get `npm install` to complete successfully. The debugging just took me a few minutes grepping package-lock.json for the 404 URL in question (https://github.com/ariya/esprima/tarball/master) and tracing that back to a top-level dependency via recursively grepping for dependent packages. I imagine that upgrading relevant dependencies might also do the trick, seeing as jshint no longer depends on esprima[0].

I'm not sure how representative this particular case is to the sort of issues you run into, but I'll tell that reproducibility issues can get a lot worse in ways that committing deps doesn't help (for example, issues like this one[1] are nasty to narrow down).

But assuming that installation in your link just happens to have a simple fix and that others are not as forgiving, how is committing node_modules supposed to help here if you're saying you can't even get it to a working state in the first place? DO you own the repo in order to be able to make the change? Or are you mostly just saying that hindsight is 20-20?

[0] https://github.com/jshint/jshint/blob/master/package.json#L4...

[1] https://github.com/node-ffi-napi/node-ffi-napi/issues/143

okk i found out another one called esprima i think i am going to use it

For those that don't know the author, Marijn Haverbeke, is the creator of CodeMirror (code editor) and later ProseMirror (text editor).

https://codemirror.net/

https://prosemirror.net/

Behind the scenes, Vrite processes the content and makes it accessible in ProseMirror-based JSON format, including the type and all the props of the Element block.

This seems to be using https://prosemirror.net

No good tool is built without using good tools, and Vrite Editor is no different. Before getting into WYSIWYG editors, I extensively researched available RTE frameworks, that could provide the tooling and functionality I was looking for. Ultimately, I picked TipTap and underlying ProseMirror — IMO, the best tools currently available for all kinds of WYSIWYG editors.

A little dissapointed to see ProseMirror not mentioned.

It's an amazing rich-text editing toolkit that provides all the bits and pieces needed to write any kind of rich-text editor. Tiptap is a wrapper over ProseMirror for minimizing the vast API surface and providing simpler configurations.

The project is using TipTap and that is mentioned.

https://prosemirror.net

The buttons had to be absolutely positioned, which required both a custom TipTap extension and tapping deeper into the underlying ProseMirror (both libraries powering the Vrite editor).

You might have noticed that the body_markdown property is set to the result of processContent() call. That’s because the Vrite API serves its content in a JSON format. Derived from the ProseMirror library powering Vrite editor, the format allows for versatile content delivery as it can be easily adapted to various needs.

Redis for cache

PHP isn’t dead. It definitely has some weirdness introduced in older versions that cannot be removed due to backward compatibility promises. However, recent versions of PHP have improved performance and developer experience significantly. Also, we use strict types and PHPStan [https://phpstan.org] (max level) to ensure type safety. And, we try to have 95%+ coverage using Pest PHP [https://pestphp.com]. With those tools, writing PHP is fun. Laravel saves a lot of time by abstracting away many HTTP, queue, and CLI-related tasks. MYSQL is the single source of truth. We sync data to Meilisearch for search. Laravel Scout makes syncing effortless. Redis is used for caching and queues.

More details on the open-source software we use are available here: [https://blogs.hyvor.com/docs/oss]

Theme Development:

In Hyvor Blogs, all themes are fully customizable. We wanted to make the theme development process as friendly as possible for developers. Being a hosted software, this is quite hard. Developers aren’t fond of (including me) editing a file on the browser to make something work. Providing an online web editor to create themes wasn’t an option. So, we created a simple CLI tool [https://github.com/hyvor/hyvor-blogs-cli] that developers can install locally via NPM. This CLI tool listens for file changes and syncs all theme files to a development blog in our production system. So, developers can make changes in their local editor and see changes with almost no delay. This has worked pretty well so far!

Theme Structure:

We wanted to keep the theme structure simple. No Javascript frameworks - just plain old-school HTML because it works the best with search engines, minimizes the data transfer required between the server and the browser, and even provides a better experience for end users.

We obviously needed a templating language to render HTML from data. There were many options like Handlebars, Liquid, and Twig. All do the job. We went with Twig because its original package is written in PHP and managed by the Symfony team so we could trust it and easily integrate it into our system.

Another thing we cared about a lot is creating standardized theme guidelines. For example, if you take WordPress themes, most themes have their own structure and are very different from each other. This adds a learning curve to each theme. To prevent that, we created standardized theme guidelines for all published themes to follow. We also standardized how common things in blogs like color theme switching, searching, language switching, etc. work. This helps users switch between and customize their themes effortlessly.

Then, there is one important thing we realized. “The structure of a blog is very simple”. First, you might think you need several stylesheets, jQuery, bootstrap, etc. NO! Just one stylesheet and barely some vanilla javascript for interactive elements like search. Realizing this helped us further improve theme performance. In our themes, the developer writes several SCSS files inside the /styles directory. This makes it easier for them to manage styles in chunks. Then, we convert all SCSS files into a single styles.css when loading it in the blog. That way, only 1 HTTP request is needed for styles - it’s super fast!

You can see more about theme development here: [https://blogs.hyvor.com/docs/themes-overview]

All official themes are free and open-source. [https://github.com/hyvor/hyvor-blogs-themes]

We have ported multiple open-source themes, and now working on a couple of original themes as well.

Caching:

We incrementally cache content using “first-request caching”. If you visit a post in the blog, the response is dynamically created and cached. Subsequent responses are served from the cache until the blogger updates the post.

This is highly efficient and scalable. Also, there is no building step involved as in Netlify or similar static hosting platforms. You can immediately see changes but also benefit from caching.

The cache is saved on a Redis server in our data centers, but we may try CDN edge caching in the future.

Multi-language support:

Multi-language support is probably the most unique selling point of Hyvor Blogs. The first version of Hyvor Blogs did not have a multi-language feature. Adding that feature took a lot of careful thought and effort, but it was totally worth it. I can safely say there’s no other hosted blogging platform that makes managing multiple languages as easy as Hyvor Blogs does.

First, we had to figure out what data was translatable. For example, post content, description, etc. Then instead of saving that data in the `posts` table, we created a new `post_variants` table to save them linked to a specific `language_id`. The blogger can create multiple languages and each entity (`post` , `tag` , `user`) can have variants for each language.

Additionally, we integrated DeepL [https://deep.com] to let bloggers automatically translate posts into many languages.

Data API filtering:

Our Data API [https://blogs.hyvor.com/docs/api-data] returns public data of the blog. This is also internally used in themes to fetch additional data. If you think about filtering data (ex: posts), one may want to filter `published_at < {time}` while another wants `published_at > {time}`. If we went with the normal API approach, we’d need many query parameters like `published_at_greater_than`, `published_at_less_than`, etc. That’s ineffective. So, we wrote a little query language called FilterQ to take a single `filter` input parameter and safely convert it to the `WHERE` part of the SQL query. With it, you can call the API with `filter=published_at>{time}` param. And, it’s even possible to use `and` / `or` and grouping for complex filtering.

Library (implemented in Laravel): https://github.com/hyvor/laravel-filterq

Sub-directory hosting:

We designed a new way to host a blog in a subdirectory of a web application. Let’s say you have a Laravel application at example.com. We created Delivery API [https://blogs.hyvor.com/docs/api-delivery] to help you host your blog at example.com/blog.

This API tells you how to deliver a response for a request (hence “Delivery” API). For example, when your Laravel app receives a request to /blog/hello-world, your app calls the Delivery API to learn how to respond to “/hello-world”. The Delivery API returns a JSON with all the data needed. Your app will then use that JSON response to create an HTTP response and send back the response to the client. It will also save the response in the cache so that it doesn’t have to call the Delivery API next time for the same path.

This is quite similar to a reverse proxy with caching, but the JSON API makes it easier to use it in web applications as we do not need HTTP parsing logic.

This is also similar to how our “first-request” caching works, but this time this caching happens inside your web application. To clear the cache, we use webhooks.

For now, we have developed libraries for Laravel and Symfony for sub-directory hosting, with plans to cover more frameworks in the future.

Rich Editor

This was probably the hardest part of all. We spent months testing many frameworks like Draft.js, Prosemirror, and even pre-built rich editors like TinyMCE. We wanted customizability and also ease-of-use. No framework checked all boxes.

We decided to go with ProseMirror [https://prosemirror.net]. It was complex but eventually, we came to understand the power of it. It has a steep learning curve, but it’s totally worth it. We actually enjoy writing Prosemirror plugins now to add some functionality to the Rich Editor. Also, recently the author added typescript support, which incredibly improved the experience. We created many nodes like Blockquotes, Callouts (with emoji), Images with captions, Embeds, and Bookmarks pretty easily after that. ProseMirror has quite good browser support as well.

Flashload

I’ve been a fan of InstantClick [http://instantclick.io/]. We wanted to add something similar to all blogs to add a “fake-fast” effect. If you haven’t used InstantClick before, it is a simple library that turns separate HTML pages into a single-page app. It starts loading content on the mouseoever event of a link and replaces the when clicked on it. This makes navigation super fast. We created an almost copy of Instantclick named Flashload [https://github.com/hyvor/flashload] with additional configurations and optimized caching. Feel free to use it in your projects :)

Overall, it’s been a great learning experience working on Hyvor Blogs. We’d love to know what HN thinks about our project. I am happy to answer any questions you might have.