AST-grep(sg) is a CLI tool for code structural search, lint, and rewriting

• ast-grep

  32 5,863 9.9 Rust

  ⚡A CLI tool for code structural search, lint and rewriting. Written in Rust

  

I really like this - it means the tool is available to people with familiarity of any of those four distribution mechanisms.

You can also download pre-built binaries from their releases page: https://github.com/ast-grep/ast-grep/releases/tag/0.14.2

On top of that, they offer API bindings for it in three different languages:

- Rust (not yet stable): https://docs.rs/ast-grep-core/latest/ast_grep_core/

- JavaScript/TypeScript: https://ast-grep.github.io/guide/api-usage/js-api.html

- Python: https://ast-grep.github.io/guide/api-usage/py-api.html

It's rare to see a tool/library offer this depth of language support out of the box.

• srgn

  5 385 9.5 Rust

  A code surgeon for precise text and code transplantation. A marriage of `tr`/`sed`, `rg` and `tree-sitter`.

Wow! What a coincidence. Just the other day I finished "v1" of a similar tool: https://github.com/alexpovel/srgn , calling it a combination of tr/sed, ripgrep and tree-sitter.

I've spent a lot of time trying to find similar tools, and even list them in the README, but `AST-grep` did not come up! I was a bit confused, as I was sure such a thing must exist already. AST-grep looks much more capable and dynamic, great work.

• InfluxDB

  www.influxdata.com sponsored

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• semgrep

  74 9,724 9.9 OCaml

  Lightweight static analysis for many languages. Find bug variants with patterns that look like source code.

  

Well, when I seach for "semgrep", I get a very nice corporate landing page with a "Book Demo" button. Which is a level of hassle that just isn't worth it for smaller teams, because "Book Demo" usually means "We're going to try to do a dance to see how much money we can extract from you." Which smaller teams may only want to do for a handful of key tools.

(4 years ago, I was more willing to put up with enterprise licensing. But in the last two years, I've seen way too many enterprise vendors try to squeeze every penny they can get from existing clients. An enterprise sales process now often means "Expect 30% annual price hikes once you're in too deep to back out.")

There's also an open source "semgrep" project here: https://github.com/semgrep/semgrep. But this seems to be basically a vulernability scanner, going by the README.

Whereas AST-grep seems to focus heavily on things like:

1. One-off searching: "Search my tree for this pattern."

2. Refactoring: "Replace this pattern with this other pattern."

AST-grep also includes a vulnerability scanning mode like semgrep.

It's possible that semgrep also has nice support for (1) and (2), but it isn't clearly visible on their corporate landing page or the first open source README I found.

• diffsitter

  15 1,517 8.7 Rust

  A tree-sitter based AST difftool to get meaningful semantic diffs

Or https://github.com/afnanenayet/diffsitter. I've tried both and I like them. No preference or notable opinions on them yet!

• ssr.nvim

  7 854 5.4 Lua

  Treesitter based structural search and replace plugin for Neovim.

There is also a neovim plugin doing structural search/replace, also based on treesitter: https://github.com/cshuaimin/ssr.nvim

• rewrite

  24 1,830 9.9 Java

  Automated mass refactoring of source code.

  

If you're into this sort of thing, there's OpenRewrite[1] for the Java ecosystem.

[1] https://docs.openrewrite.org/

• pfff

  6 2,422 0.0 OCaml

  Discontinued Tools for code analysis, visualizations, or style-preserving source transformation.

  

Hi, ast-grep author here. This is a great question and I asked this in the first place before I started the hobby project.

TLDR; I designed ast-grep to be on different tracks than semgrep.

Semgrep is for security and ast-grep is for development.

First and foremost, I have always been in awe of semgrep. Semgrep's documentation, product sites and Padioleau's podcast all gave me a lot of inspiration. Using code to find code is such a cool idea that I never need to craft an intricate regex or write a lengthy AST program. sgrep and patch from https://github.com/facebookarchive/pfff/wiki/Sgrep have helped me a lot in real large codebases.

When I used semgrep as a software engineer, instead of a security researcher, I found semgrep has not touched too much on routine development works. I can use `semgrep -e PATTERN` but the Python wrapper is not too fast compared to grep.

• WorkOS

  workos.com sponsored

  The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

  

• cgrep

  1 0 6.4 TypeScript

  cgrep is an npm package for writing project checks with regex.

  

Also plugging my related project: https://github.com/Ichigo-Labs/cgrep

• ruff

  95 26,504 10.0 Rust

  An extremely fast Python linter and code formatter, written in Rust.

  

I confess I stole the pip recipe from Charlie :D

https://github.com/astral-sh/ruff/blob/main/.github/workflow...

• contract

  1 2 9.7 C#
  

Interesting use-case: We use ast-grep extensively at CodeRabbit[0] to review pull requests with AI.

We use gpt-4 to generate ast-grep patterns to deep-dive and verify pull-request integrity. We just rolled this feature out 3 days back and we are getting amazing results!

For example, comments such as these are powered by AI-generate ast-grep queries: https://github.com/amorphie/contract/pull/100#discussion_r14...

[0]: https://coderabbit.ai

• gram_grep

  4 11 7.1 C++

  Search text using a grammar, lexer, or straight regex. Chain searches for greater refinement.

There is also gram_grep[0]"Search text using a grammar, lexer, or straight regex. Chain searches for greater refinement."

See also parsertl-playground[1] for online edit/test grammars.

[0]https://github.com/BenHanson/gram_grep

• oak

  2 6 8.0 Go

  GO GO PARSE YOUR CODE GO GO (by go-go-golems)

  

I'll post my own crappy one called oak which uses templates to render the result of tree-sitter queries.

https://github.com/go-go-golems/oak

I initially hope the queries would be more powerful, but they are really not. You can write queries and a resulting template in a yaml file. The program will scan a list of repositories for all these YAML files, and expose them as command line verbs.

Here is one to find go definitions:

https://github.com/go-go-golems/oak/blob/main/cmd/oak/querie...

This can then be run as:

         oak go definitions /home/manuel/code/wesen/corporate-headquarters/geppetto/pkg/cmds/cmd.go

• syntax-searcher

  1 4 6.6 Rust

  Language-independent command-line utility for syntax-aware pattern matching.

Most other repositories print all results pretty much instantly.

[1] https://github.com/jgke/syntax-searcher

Suggest a related project