shiki VS tree-sitter

Shiki (https://shiki.style) might be good enough for your usecase, use the latest beta versions as it's the new fresh esm rewrite.

Finally the vanguard had arrived. A markdown renderer named Shiki.

Can I introduce you to our lord and savior Shiki? https://github.com/shikijs/shiki

Coincidentally, I recently looked into the slow startup time for shiki and it was mostly from parsing JSONC, not WASM: https://github.com/shikijs/shiki/issues/439.

Warning: This article refers to versions of Code Hike before 0.8.0. Since version 0.8.0, Code Hike uses lighter instead of Shiki. The approach presented in this article uses shiki's color replacement method, which no longer works with lighter.

Prismjs was replaced by Shiki and rehype-pretty-code.

The outcome I was (mostly) hoping for has been done by Shiki library! I take no credit for the idea btw, I am just glad that someone else (Pine) took it on!

I learned from a google search that these days upstream tree-sitter provides WebAssembly bindings.

Source: https://github.com/tree-sitter/tree-sitter/tree/master/lib/b...

NPM: https://www.npmjs.com/package/web-tree-sitter

Download from the latest Github release: js file (https://github.com/tree-sitter/tree-sitter/releases/download...) and wasm file (https://github.com/tree-sitter/tree-sitter/releases/download...)

Tree-sitter optimizes for performance (to use in editors), not for correctness. In fact even TS' core developers advocate for not bothering too much with correctness of grammars[1]. I imagine this constraint would be a deal-breaker for GitHub or anyone else in their position.

[1] https://github.com/tree-sitter/tree-sitter/issues/130#issuec...

This is a plugin that provides a simple way to use the tree-sitter in Neovim and also provides functionalities like highlighting, etc.

From the tree-sitter side, I am tracking https://github.com/tree-sitter/tree-sitter/issues/1942

Is tree-sitter really slower than TextMate grammars? Some benchmarks indicate that this isn't really the case [1]. On the other hand, breaking parse trees is a real issue, because the error-recovery in tree-sitter is pretty rudimentary [2][3], but as you said, it's not an issue for Shiki.

Several TextMate grammars suffer from inaccuracy bugs, and issues of maintainability. Perhaps the biggest hindrance in the adoption of tree-sitter, is that the most popular editor, VSCode, still doesn't support it.

[1]: https://github.com/microsoft/vscode/pull/161479

[2]: https://github.com/tree-sitter/tree-sitter/issues/1870

[3]: https://github.com/tree-sitter/tree-sitter/issues/224

u/lewis6991 I think the biggest performance gain was made by tree-sitter itself: https://github.com/tree-sitter/tree-sitter/pull/2085

I asked on the repo's discussions but responses are limited and not explanatory (I'm not shaming anyone here, discussions aren't a place for detailed how-tos and documentations anyway).

The following discussion here. TS query cannot be incremental, that is why I regard it as design fault.

Hi, so I've recently decided to give Neovim yet another try, this time using some predefined plugins with kickstart.nvim, for syntax it uses tree-sitter.