refex
semgrep
refex | semgrep | |
---|---|---|
3 | 75 | |
14 | 9,742 | |
- | 1.4% | |
5.0 | 9.9 | |
7 months ago | 3 days ago | |
Python | OCaml | |
Apache License 2.0 | GNU Lesser General Public License v3.0 only |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
refex
-
Writing and Linting Python at Scale
As someone who worked on a similar tool (https://github.com/ssbr/refex/tree/main/refex/fix/fixers, I did a bunch of the work to prep this for open-sourcing, though I think all my contributions are hidden behind the "Google-internal" anonymization), having auto-applied or auto-appliable fixers like this is super useful.
They can be auto-applied by post-commit (e.g. a generic `git fixcommit` style command that runs all the relevant lint tools and fixes them in the working copy, letting you review before push), or applied during code review (automatic comments with a "click here to apply fix" interface), both of which are nice.
Plus the same underlying tooling can be used to write more complex one-off fixes that may be used for migrations or cleanups.
-
I learnt to use ASTs to patch 100,000s lines of python code
You might like Refex, which automates AST transformations like this:
-
Semgrep: Like Grep but for Code
There's lots of confusion about what semgrep does here, which is kind of unfortunate. I haven't touched it much, but I have built a very similar tool (I'm one of the contributors to refex[1], which is a very similar project).
The starting point of semantic grep is very useful. When you have a big codebase, you often want to detect antipatterns, or not even antipatterns, but just uses of a thing, say you're renaming a method and want to track down the callers.
Being able to act on the AST, instead of hoping you searched up all of the variants of whitespace and line breaks and, depending on the specific example, different uses of argument passing, is really useful.
But often when you're semantically grepping, your goal is to replace something with something else (this is what refex was initially built for: to aide in large scale changes in python, as a sort of equivalent to the C++ tools that Google uses).
But then you want to shift left even further: once you have a pattern that you want to replace once, you can just enforce that a linter yell at you when anyone does it again. So it's very natural to develop a linter-style thing on top of one of these[2].
This is, as I understand it sort of the same thing that happens in C++: clang-tidy and clang-format are written on top of AST libraries that can be used for ad-hoc analysis and transformations, but you can also just plug them into a linter.
The thing is, for most organizations, enforcing code style and best practices is more valuable than apply a refactoring to 10M lines of code, because most organizations don't have 10M lines of code to refactor. That doesn't mean that these tools aren't also useful for ad-hoc transforms and exploratory analysis. They absolutely are!
[1]: https://github.com/ssbr/refex
[2]: https://github.com/ssbr/refex/tree/main/refex/fix
semgrep
- Semgrep: Semantic Grep for Code
-
A Deep Dive Into Terraform Static Code Analysis Tools: Features and Comparisons
Semgrep OSS Owner/Maintainer: Semgrep Age: First release on GitHub on February 6th, 2020 License: GNU Lesser General Public License v2.1
- Semgrep – Find bugs and enforce code standards
-
Application Security - Bridging Frontend and Cybersecurity: What is Application Security?
Semgrep - https://semgrep.dev
-
Creating a DevSecOps pipeline with Jenkins — Part 1
For the SAST stage, I used SonarQube tool. SonarQube is an open-source platform developed by SonarSource for continuous inspection of code quality to perform automatic reviews with static analysis of code to detect bugs and code smells on more than 30 programming languages. I preferred SonarQube instead of other SAST tools because it has a detailed documentation and plugins about integration with Jenkins and SonarQube works with Java projects pretty well. Of course you can similar multi-language-supported tools such as Semgrep or language-specific tools such as Bandit.
-
Tree-Sitter
> Not sure I understand your point.
The problem is using Treesitter (for syntax highlighting and "semantic movements") and an LSP at the same time. So if your language has a LSP, using Treesitter additionally is redundant at best and introduces inconcistency at worst.
I'm not talking about using Treesitter as the parser for the LSP.
> Most popular languages have language-specific tools
I'd say even less popular langauges like Coq^H^H^HRocq, Lean 4, Koka, Idris, Unison, ... have their "own" tools, I do not know of a language that uses a Treesitter parser in its LSP, but I do know about tools like https://semgrep.dev/ (written in OCaml) and Github's code search which use Treesitter.
-
AST-grep(sg) is a CLI tool for code structural search, lint, and rewriting
Well, when I seach for "semgrep", I get a very nice corporate landing page with a "Book Demo" button. Which is a level of hassle that just isn't worth it for smaller teams, because "Book Demo" usually means "We're going to try to do a dance to see how much money we can extract from you." Which smaller teams may only want to do for a handful of key tools.
(4 years ago, I was more willing to put up with enterprise licensing. But in the last two years, I've seen way too many enterprise vendors try to squeeze every penny they can get from existing clients. An enterprise sales process now often means "Expect 30% annual price hikes once you're in too deep to back out.")
There's also an open source "semgrep" project here: https://github.com/semgrep/semgrep. But this seems to be basically a vulernability scanner, going by the README.
Whereas AST-grep seems to focus heavily on things like:
1. One-off searching: "Search my tree for this pattern."
2. Refactoring: "Replace this pattern with this other pattern."
AST-grep also includes a vulnerability scanning mode like semgrep.
It's possible that semgrep also has nice support for (1) and (2), but it isn't clearly visible on their corporate landing page or the first open source README I found.
-
Top 10 Snyk Alternatives for Code Security
7. Semgrep
-
semgrep VS bearer - a user suggested alternative
2 projects | 10 Jul 2023
What are some alternatives?
pre-commit - A framework for managing and maintaining multi-language pre-commit hooks.
SonarQube - Continuous Inspection
checkr - Custom static analysis rules for the lazy. Write project specific static analysis checks in a few lines of code.
snyk - Snyk CLI scans and monitors your projects for security vulnerabilities. [Moved to: https://github.com/snyk/cli]
tree-sitter-swift - Swift grammar for tree-sitter
codeql - CodeQL: the libraries and queries that power security researchers around the world, as well as code scanning in GitHub Advanced Security
ocaml-tree-sitter-semgrep - Generate parsers from tree-sitter grammars extended to support Semgrep patterns
Spotbugs - SpotBugs is FindBugs' successor. A tool for static analysis to look for bugs in Java code.
CCGrep - Code Clone Detector like grep
Bear - Bear is a tool that generates a compilation database for clang tooling.
detect-secrets - An enterprise friendly way of detecting and preventing secrets in code.