tree-sitter-c VS PHP Parser

(use-package treesit :ensure nil :custom ;; Some stuff taken from here: https://robbmann.io/posts/emacs-treesit-auto/ (treesit-extra-load-path '("/usr/lib64/")) (treesit-language-source-alist '((bash . ("https://github.com/tree-sitter/tree-sitter-bash")) (c . ("https://github.com/tree-sitter/tree-sitter-c")) (c++ . ("https://github.com/tree-sitter/tree-sitter-cpp")) (csharp . ("https://github.com/tree-sitter/tree-sitter-c-sharp")) (css . ("https://github.com/tree-sitter/tree-sitter-css")) (elixir ("https://github.com/elixir-lang/tree-sitter-elixir")) (html . ("https://github.com/tree-sitter/tree-sitter-html")) (java . ("https://github.com/tree-sitter/tree-sitter-java")) (javascript . ("https://github.com/tree-sitter/tree-sitter-javascript")) (json . ("https://github.com/tree-sitter/tree-sitter-json")) (lua . ("https://github.com/Azganoth/tree-sitter-lua")) (makefile . ("https://github.com/alemuller/tree-sitter-make")) (org . ("https://github.com/milisims/tree-sitter-org")) (python . ("https://github.com/tree-sitter/tree-sitter-python")) (tsx . ("https://github.com/tree-sitter/tree-sitter-typescript" "master" "tsx/src")) (typescript . ("https://github.com/tree-sitter/tree-sitter-typescript" "master" "typescript/src")) (ruby . ("https://github.com/tree-sitter/tree-sitter-ruby")) (rust . ("https://github.com/tree-sitter/tree-sitter-rust")) (sql . ("https://github.com/m-novikov/tree-sitter-sql")) (toml . ("https://github.com/tree-sitter/tree-sitter-toml")) (yaml . ("https://github.com/ikatyang/tree-sitter-yaml")))) (major-mode-remap-alist '((c-mode . c-ts-mode) (c++-mode . c++-ts-mode) (csharp-mode . csharp-ts-mode) (css-mode . css-ts-mode) (html-mode . html-ts-mode) (java-mode . java-ts-mode) (js-mode . js-ts-mode) (json-mode . json-ts-mode) (makefile-mode . makefile-ts-mode) ;; (org-mode . org-ts-mode) ;; not mature yet (python-mode . python-ts-mode) (typescript-mode . typescript-ts-mode) (ruby-mode . ruby-ts-mode) (rust-mode . rust-ts-mode) (toml-mode . toml-ts-mode) (yaml-mode . yaml-ts-mode))) (treesit-auto-fallback-alist '((toml-ts-mode . conf-toml-mode) (typescript-ts-mode . nil) (tsx-ts-mode . nil))) (treesit-font-lock-settings t) (treesit-simple-indent t) (treesit-defun-type-regexp t)) (use-package treesit-auto :demand t :config (setq treesit-auto-install t) (global-treesit-auto-mode))

The feature isn't working well yet on C and C++. If I recall correctly it's based on Tree-Sitter[1] parsing, and there are still too many bugs in corresponding grammars - tree-sitter-c[2] and tree-sitter-cpp[3]. Hopefully, it will be greatly improved in the future as the share of the existing and newly written code in C and C++ is quite significant.

[1] https://tree-sitter.github.io/tree-sitter/

[2] https://github.com/tree-sitter/tree-sitter-c/issues

[3] https://github.com/tree-sitter/tree-sitter-cpp/issues

[[language]] name = "c" scope = "source.c" injection-regex = "c" file-types = ["c"] # TODO: ["h"] roots = [] comment-token = "//" language-server = { command = "clangd" } indent = { tab-width = 8, unit = " " } [language.debugger] name = "lldb-vscode" transport = "stdio" command = "lldb-vscode" [[language.debugger.templates]] name = "binary" request = "launch" completion = [ { name = "binary", completion = "filename" } ] args = { console = "internalConsole", program = "{0}" } [[language.debugger.templates]] name = "attach" request = "attach" completion = [ "pid" ] args = { console = "internalConsole", pid = "{0}" } [[language.debugger.templates]] name = "gdbserver attach" request = "attach" completion = [ { name = "lldb connect url", default = "connect://localhost:3333" }, { name = "file", completion = "filename" }, "pid" ] args = { console = "internalConsole", attachCommands = [ "platform select remote-gdb-server", "platform connect {0}", "file {1}", "attach {2}" ] } [[grammar]] name = "c" source = { git = "https://github.com/tree-sitter/tree-sitter-c", rev = "7175a6dd5fc1cee660dce6fe23f6043d75af424a" }

There's no way to highlight format specs currently, since the treesitter C grammar doesn't specify them. You could open an issue on the tree-sitter-c repo.

[1] https://github.com/tree-sitter/tree-sitter-c/issues/51

Interesting. One of the reasons I stopped considering Rector is because of how memory, CPU, and time intensive it is for a non-trivial project. Instead I've been using Nikita's PHP Parser directly and getting much better results even though it isn't multi-threaded out of the box.

nikic/PHP-Parser uses a Bison equivalent for PHP parsing. See the grammar file https://github.com/nikic/PHP-Parser/blob/4.x/grammar/php7.y.

Maybe nikic/PHP-Parser is an alternative. If you only need to tokenizer part, PHP has an extension too.

nikic/PHP-Parser

Code parsing. Many linters and code builders use php-parser. It uses YACC (analog Bison) to build AST.

The PR in PHP-Parser is already on the way ↓ https://github.com/nikic/PHP-Parser/pull/834

I see there is the https://github.com/nikic/PHP-Parser

It's good idea. But something strange. Php was made as language which embedded to HTML. And how will it work without open and close tags? Instead of parsing and replacing source code as string you might use parser e.g https://github.com/nikic/PHP-Parser

I wish there was a more universal format for parsers, but I just don't think there enough people who know their stuff.

Take PHP, a language that a lot of people use: the tree-sitter-php extension doesn't support features added in 2019, let alone features added towards the end of 2020.

If you want an up-to-date PHP parser, there's really only one open-source parser[0] that's accurate enough to be used on PHP codebases old and new, and it's written in PHP. Then if you want to parse in a robust fashion you have to adopt a number of hacks to get everything working.

I hadn't encountered LSIF before – can GitHub be configured to use those maps?

[0] https://github.com/nikic/PHP-Parser