tower-lsp VS tree-sitter

I realize this might not be for everyone, but I'm writing it in Rust using Lib-ruby-parser and tower-lsp: two existing libraries that handle a bunch of the heavy lifting for me. I'm more productive in Rust than with Ruby at this point, despite doing Ruby full time for 15 years, plus I really really don't want to have to deal with a slow LSP--that was the whome impetus for this project. I started in the spring, made a bunch of headway, then backtracked to redo the internals to make it easier to handle monkeypatching, overriding/redefining of methods, etc. across your project.

https://github.com/ebkalderon/tower-lsp is a generalized LSP implementation in a lower-level language (Rust) so you may get a better idea by reading through that repo. It seems that the server opens a TCP socket that the client later connects to, but I'm not really sure.

A LSP is just an api implemented to Microsoft's LSP spec. You can implement that API however you wish but something like tower-lsp can do a lot of the heavy lifting for you.

Mine all use [tower-lsp](https://github.com/ebkalderon/tower-lsp/) for the LSP protocol stuff, and then either [Tree-sitter](https://github.com/tree-sitter/tree-sitter) or [Nom](https://github.com/Geal/nom). If I do another I'll probably try [Chumsky](https://github.com/zesterer/chumsky) which combines some of the advantages of both.

From the tree-sitter side, I am tracking https://github.com/tree-sitter/tree-sitter/issues/1942

We rely on the Rust bindings from the main tree-sitter repository on the C library. Loading and matching the queries on the source file is done by this library, and so out of scope for Topiary.

Is tree-sitter really slower than TextMate grammars? Some benchmarks indicate that this isn't really the case [1]. On the other hand, breaking parse trees is a real issue, because the error-recovery in tree-sitter is pretty rudimentary [2][3], but as you said, it's not an issue for Shiki.

Several TextMate grammars suffer from inaccuracy bugs, and issues of maintainability. Perhaps the biggest hindrance in the adoption of tree-sitter, is that the most popular editor, VSCode, still doesn't support it.

[1]: https://github.com/microsoft/vscode/pull/161479

[2]: https://github.com/tree-sitter/tree-sitter/issues/1870

[3]: https://github.com/tree-sitter/tree-sitter/issues/224

Tree-sitter is going to be a lot slower to load: https://github.com/tree-sitter/tree-sitter/issues/1942.

It's ok for things like editor but using it in a SSG for example is hard when just loading the syntaxes will take multiple times the time it takes to render the whole thing with a regex based highlighter.

u/lewis6991 I think the biggest performance gain was made by tree-sitter itself: https://github.com/tree-sitter/tree-sitter/pull/2085

I asked on the repo's discussions but responses are limited and not explanatory (I'm not shaming anyone here, discussions aren't a place for detailed how-tos and documentations anyway).

The following discussion here. TS query cannot be incremental, that is why I regard it as design fault.

Hi, so I've recently decided to give Neovim yet another try, this time using some predefined plugins with kickstart.nvim, for syntax it uses tree-sitter.

Another thing that comes to my mind is Tree-sitter. In my understanding it generates syntax trees that can not only be used for syntax highlighting but also for building advanced text manipulation in plug-ins. Maybe it can save you from implementing the parsing in the first place. There is already a Latex-Grammar and a rust create to use treesitter.

What do you think about treesitter? https://github.com/tree-sitter/tree-sitter

The idea is to sync changes in the text to a tree structure, then have all the structure manipulation functions built on top of it. See the gif here for a visual representation: https://github.com/nvim-treesitter/playground