ruby-lsp VS prism

Some details from https://github.com/Shopify/ruby-lsp

> NOTE: starting with v0.7.0, it is no longer recommended to add the ruby-lsp to the bundle. The gem will generate a custom bundle in .ruby-lsp/Gemfile which is used to identify the versions of dependencies that should be used for the application (e.g.: the correct RuboCop version).

After lunch, I parked at the Ruby LSP table. I worked with Julie and Drew on HEREDOC endings. We also found an issue that we spent some time digging into, but ran out of time before we had a solution. It was fun spelunking through the code with friends. Ufuk joined us in looking at the cursor indentation issue. His sage wisdom helped us navigate through an unfamiliar codebase.

Thanks for the feedback. Added an issue for you: https://github.com/Shopify/ruby-lsp/issues/594

Links: Ruby-lsp , vscode addon

Thanks. You might like to watch this RFC PR https://github.com/Shopify/ruby-lsp/pull/429 (from me) with the proof of concept implementation of indexing and a very simplistic go to definition implementation, which is now being productionized properly. The indexing logic has moved into Syntax Tree and now we have access to a uniform API in Ruby LSP.

Ruby LSP and its associated extension: https://github.com/Shopify/ruby-lsp

IRB type completion comes as a result of a chain of events which starts from the incredible work done by Kevin Newton (et al) to write a new canonical Ruby parser called Prism in C99 with no dependencies [1].

With Prism, you can then create tool suites like syntax_tree [2], which then leads Prettier formatters [3], a new Ruby LSP [4], which unlocks a new Ruby LSP VS Code extension [5], not to mention a laundry list of other gems like Rubocop and of course Ruby itself that will benefit from a faster and more maintainable Ruby parser.

It's a beautiful illustration of the power of questioning conventions, going back to first principles to uncover better solutions to previously solved problems, whose new solutions create new capabilities which unlocks the ability to solve new problems.

[1]: https://github.com/ruby/prism

Kevin Newton talked about Prism, a new Ruby parser. He discussed the challenges that come with parsing Ruby. He shared what's next, and what we can and should expect from our Ruby tooling in the future. He ended with an impassioned discussion about building a contributor community around a single tool.

It’s disappointing there doesn’t seem to be an easily available blog post or announcement (maybe I just didn’t find it), but the design doc lists a few motivations: https://github.com/ruby/prism/blob/main/docs/design.md

It looks like there was also a podcast interview last year that touches on the origins of the project: https://topenddevs.com/podcasts/ruby-rogues/episodes/the-new...

Reading between the lines it looks to me like this is motivated by ripper (the old parser) not being a great fit for tooling around ruby like IDE LSP integrations and such. Ripper isn’t fault tolerant (if the script has a a syntax error you don’t get a partial tree, just an exception); being implemented in ruby enough itself that it kind of depends on ruby which isn’t always convenient for integration (IDEs like vscode make plugins in JS easy, prism comes with node bindings), and maybe being enough of a crufty old code base that maintaining it and fixing those design issues was deemed impracticable.

Also worth noting if it wasn’t clear I’m pretty sure this parser is not being used or intended to be used for a ruby runtime to actually execute scripts, and that’s not what ripper was for either. This is for tooling that operates on ruby files for other purposes: syntax highlighting, linting, stuff like that.

I thought yarp was where ruby parsing was heading