deptry VS sqlparse

deptry – Find unused, missing and transitive dependencies in a Python project

deptry – A command line utility to check for obsolete, missing and transitive dependencies in a Python project

We are happy to share that deptry 0.10.0 has been released! Deptry is a command line tool to check for issues with dependencies in a Python project, such as obsolete or missing dependencies.

In this latest release, Some major improvements were added to the way deptry reports issues by [Mathieu Kniewallner](https://github.com/mkniewallner). You can find the full release notes [here](https://github.com/fpgmaas/deptry/releases/tag/0.10.0).

If you're interested in learning more about deptry, be sure to check out the [Documentation](https://fpgmaas.github.io/deptry/) and the [GitHub repository](https://github.com/fpgmaas/deptry).

Let us know if you have any questions or feedback!

Since PEP-621 does not specify a recommended way to define development dependencies, everything is expected to be a regular dependency. See here.

Thanks for your comment :) src was used purely as an example. By default, the tool scans for .py files in all directories recursively. But for example, in this issue someone put their source code in crop directory and thus called the tool with deptry crop/, which is not how the argument is supposed to be used.

Your project could use some additional documentation. Now the only way for me to find out how to use it is through the 'open colab' button. You could consider adding an example to the README. I personally always try to add a documentation website, which is really easily done with e.g. mkdocs or Sphinx. For an example, you could check out my most recent project deptry.

I have recently been working on a project called `deptry`, a command line tool to check for issues in the dependencies of Python projects. It can be used to find obsolete, missing, transitive and misplaced development dependencies. It supports the following types of projects:

- Projects that use Poetry and a corresponding pyproject.toml file

- Projects that use a requirements.txt file according to the pip standards

---

* Documentation: https://fpgmaas.github.io/deptry/

* GitHub repository: https://github.com/fpgmaas/deptry

---

I am quite happy with the project in its current form, but I also realise there is still a lot of room left for improvement. Therefore, I hope some people are willing to give it a try and provide me with feedback. So; if you have a project with a long list of dependencies and a little bit of spare time on your hands, please give it a try and let me know what you think!

If you encounter any issues, find a bug, or have any other form of feedback, please don't hesitate to raise an issue in the GitHub repository, or leave a comment here.

Kind regards,

Florian

P.S. Many thanks to Hirokazu Takaya (https://github.com/lisphilar) for incorporating it in the CI/CD pipeline of his project covid19-sir (https://github.com/lisphilar/covid19-sir). It provided me with very valuable early feedback.

- Projects that use a _requirements.txt_ file according to the [pip](https://pip.pypa.io/en/stable/user_guide/) standards

* [*Documentation*](https://fpgmaas.github.io/deptry/)

Suggest checking out the sqlparse library for a way to do the different flavours without needing to address each case directly: https://github.com/andialbrecht/sqlparse

Gotcha, since we haven't actually written all of this yet I don't have any useful code snippets to share but we've discussed tackling the problem internally using something like sqlparse. You'd need to identify the relevant sql chunks, parse them for table dependency information and then create the relevant entities in whichever data lineage tool you were using.

sqlparse – A non-validating SQL parser module for Python

Regular expressions is a popular approach to extract information from SQL statements. However, regular expressions quickly become too complex to handle common features like WITH, sub-queries, windows clauses, aliases and quotes. sqlparse is a popular python package that uses regular expressions to parse SQL.

This one is not bad: https://github.com/andialbrecht/sqlparse.

We used this repo for this: https://github.com/andialbrecht/sqlparse. I may have miscommunicated. We didn't write the parser from scratch, we created a way for the parser to detect downstream and upstream dependencies of the resource.