parser VS sourcegraph

I have mixed feelings on Rust's syntax, especially around generics, lifetimes, and the `modifier -> keyword` syntax (i.e. `async fn` or `pub fn`). For Inko, I wanted something that's easy to parse by hand, and no context specific parsing (e.g. `QUOTE -> something` being the start of a lifetime in one place, but a char literal in another place).

Another motivator for that is that years ago I worked on Rubinius for a while (an implementation of Ruby), and helped out with a parser for Ruby (https://github.com/whitequark/parser). The Ruby developers really liked changing their already impossible syntax in even more impossible ways on a regular basis, making it a real challenge to provide syntax related tools that support multiple Ruby versions. I wanted to avoid making the same mistake with Inko, hence I'm actively trying to keep the syntax as simple as is reasonable.

As for the specific examples:

- `fn async` means your parser only needs to look for `A | B | fn` in a certain scope, instead of `A | B | fn | async fn`. This cuts down the amount of repetition in the parser. An example is found at https://github.com/inko-lang/inko/blob/8f5ad1e56756fe00325a3..., which parses the body of a class definition.

- Skipping parentheses is directly lifted from Ruby, because I really like it. Older versions took this further by also letting you write `function arg1 arg2`, but I got rid of that to make parsing easier. It's especially nice so you can do things like `if foo.bar.baz? { ... }` instead of `if foo().bar().baz?()`, though I suspect opinions will differ on this :)

- Until recently we did in fact use `::` as a namespace separator, but I changed that to `.` to keep things consistent with the call syntax, and because it removes the need for remembering "Oh for namespaces I need to use ::, but for calls .".

- `[T]` for generics is because most editors automatically insert a closing `]` if you type `[`, but not when you type `<`. If they do, then trying to write `10<20` is annoying because you'd end up with `10<>20`. I also just like the way it looks more. The usual ambiguity issues surrounding `<>` (e.g. what leads to `foo::()` in Rust) doesn't apply to Inko, because we don't allow generics in expressions (i.e. `Array[Int].with_capacity(42)` isn't valid syntax) in the first place.

So there was this really awesome gem called parser written by someone not on the core team that gives you a super clean understanding of the Ruby code. Not only does it not care if the parentheses are there or not, but there's a really well structured and precise mapping of where the information comes from and it is completely semantic. So if you've got parentheses or not, it's not gonna make any difference in the structure of your abstract syntax tree, but you can actually ask where are the locations. That is taken care of, but the understanding of the code, what's going on in the code is completely independent of if you wrote those parentheses or not.

Working with those things, possibly with the help of reading books, tends to be how it's learned I'd say. I'm not the one you asked, but I personally worked with Ruby for 10 years, worked on a system to improve coverage reports, which relied on rewriting ruby code. Doing so was done using the Parser gem, which is a ruby parser that has a different abstract syntax tree (https://github.com/whitequark/parser). I'm also interested in programming languages development, so I try to read on this / develop my own language in my free time.

Changes like this have been going on for years. I remember that back when I was still helping out with https://github.com/whitequark/parser, the author on a regular basis had to deal with Ruby making yet more non-trivial syntax changes. IIRC they eventually burned out on the project because of that, but my memory is a bit fuzzy.

This is more a function of Ruby than of tree-sitter. The tree-sitter grammars for other languages are hopefully less inscrutable. For Ruby, we basically just ported whitequark's parser [1] over to tree-sitter's grammar DSL and scanner API.

[1] https://github.com/whitequark/parser

Sourcegraph | REMOTE | Full-Time | Machine Learning Engineer, Developer Advocate, Enterprise Product Manager, Technical Advisor | https://sourcegraph.com

Sourcegraph is a code AI platform that makes it easy to read, write, and fix code–even in big, complex codebases.

We are building Cody, an AI coding assistant that uses code search and code intelligence to help devs quickly understand what's happening in code and generate new code that matches the best practices in your codebase. Cody supports AI-enabled autocompletion, fixing bugs, refactoring, test generation, code explanation, and answering high-level questions. You can read Steve Yegge's post on why Cody's code context engine differentiates it from the fast-moving field of AI dev tools: https://about.sourcegraph.com/blog/cheating-is-all-you-need.

Apply here: https://grnh.se/0572f98b4us

That's pretty much what https://sourcegraph.com/ are selling, is it not?

Despite their shitty rug-pull <https://github.com/sourcegraph/sourcegraph/pull/53345>, I do really like Sourcegraph and one doesn't (currently?!) need to be logged in to use it: https://sourcegraph.com/search and they have a handy rewrite pattern such that one can just plug the repo path into the URL for quick searching e.g. https://sourcegraph.com/github.com/JetBrains/intellij-commun...

- https://sourcegraph.com is pivoting and building a copilot application (named Cody). This is pretty good, since sourcegraph is great at understanding your code

While a readable Dockerfile can work as documentation, there are a few caveats:

* the application needs to be designed to work outside containers (so, no hardcoded URLs, ports, or paths). Also, not directly related to containers, but it's nice if it can be easily compiled in most environments and not just on the base image.

* I still need a way to notify me of updates; if the Dockerfile just wgets a binary, this doesn't help me.

* The Dockerfiles need to be easy to find. Sourcegraph's don't seem to be referenced from the documentation, I had to look through their Github repos to find https://github.com/sourcegraph/sourcegraph/tree/main/docker-... (though most are bazel scripts instead of Dockerfiles, but serve the same purpose)

We use Sourcegraph, which is a tool that searches through code in repositories. We leverage this tool in order to understand the adoption curve of our components across all of Reddit. We have a dashboard for each of the platforms to compare the inclusion of RPL components over legacy components. These insights are helpful for us to make informed decisions on how we continue to drive RPL adoption. We love seeing the green line go up and the red line go down!

SourceGraph: https://github.com/sourcegraph/sourcegraph/pulls?q=is%3Apr+a...