tbpatch
LibCST
| tbpatch | LibCST | |
|---|---|---|
| 1 | 9 | |
| 0 | 1,430 | |
| - | 2.5% | |
| 0.0 | 8.5 | |
| about 3 years ago | 1 day ago | |
| Rust | Python | |
| MIT License | GNU General Public License v3.0 or later |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
tbpatch
-
Our Engineering Team Used Python's AST to Patch 100,000s of Lines of Code
I had a problem of applying patches to the files that got indented differently meantime.
So I wrote a tool that takes a context diff, rebuilds the before and after text, then parses both as vectors of (leading whitespaces, token), then creates a diff of those vectors ignoring the whitespaces, parses the target file as a similar vector, and applies this diff to such a vector (again with ignoring/copying whitespaces) - after which it reconstructs the text.
The results are surprisingly robust, despite the relatively naive tokenization heuristic.
https://github.com/ayourtch/tbpatch
LibCST
- Show HN: Codemodder – A new codemod library for Java and Python
-
Package that graphs and exports jpeg of CST/AST?
LibCST: Seems to only show in terminal.
-
How to approach modifying source code programmatically?
While you can do this using ANTLR or any other lexing/parsing tool, it's honestly a bit of a pain. Whitespace and comments can go almost anywhere, even in the middle of expressions, so the grammar ends up becoming fairly messy. So, I'd recommend using a library that handles this for you, if at all possible. For example, if I wanted to code-mod Python I'd prob just use the LibCST library.
-
ruff is a fast Python linter written in Rust
I recommend https://github.com/Instagram/LibCST (which is currently implementing rust bindings)
-
How to handle line endings when writing files depending on OS?
I've been roughly copying some of the logic from the LibCST project. This struct in particular- https://github.com/Instagram/LibCST/blob/main/native/libcst/src/tokenizer/text_position/char_width.rs does a good job of normalizing the line endings of a str. The long way around you could mimic this construct, transform the str to normalized line endings, and then split on "\n" or make a somewhat more complicated transformer which turns a large str into a Vec (or Vec).
-
We're the core team behind the popular Python autoformatter: Black. AMA!
I myself am working on upgrading LibCST's parser engine to support the new syntax, and then am hoping we can rewrite Black's formatting rules in terms of LibCST's API. That's not a small amount of work, which is why we can't confidently say that's going to be the way forward.
-
Our Engineering Team Used Python's AST to Patch 100,000s of Lines of Code
Never used it but it appears that Facebook/Instagram have a format preserving CST library for Python: https://github.com/Instagram/LibCST
What are some alternatives?
pasta - Library to refactor python code through AST manipulation.
RedBaron - Bottom-up approach to refactoring in python
Bowler - Safe code refactoring for modern Python.
lib3to6 - Build universally compatible python packages from a substantial subset of Python 3.8.
black - The uncompromising Python code formatter
ufmt - Safe, atomic formatting with black and µsort
instaviz - Instant visualization of Python AST and Code Objects
ruff - An extremely fast Python linter and code formatter, written in Rust.