deptry VS docquery

deptry – Find unused, missing and transitive dependencies in a Python project

deptry – A command line utility to check for obsolete, missing and transitive dependencies in a Python project

We are happy to share that deptry 0.10.0 has been released! Deptry is a command line tool to check for issues with dependencies in a Python project, such as obsolete or missing dependencies.

In this latest release, Some major improvements were added to the way deptry reports issues by [Mathieu Kniewallner](https://github.com/mkniewallner). You can find the full release notes [here](https://github.com/fpgmaas/deptry/releases/tag/0.10.0).

If you're interested in learning more about deptry, be sure to check out the [Documentation](https://fpgmaas.github.io/deptry/) and the [GitHub repository](https://github.com/fpgmaas/deptry).

Let us know if you have any questions or feedback!

Since PEP-621 does not specify a recommended way to define development dependencies, everything is expected to be a regular dependency. See here.

Thanks for your comment :) src was used purely as an example. By default, the tool scans for .py files in all directories recursively. But for example, in this issue someone put their source code in crop directory and thus called the tool with deptry crop/, which is not how the argument is supposed to be used.

Your project could use some additional documentation. Now the only way for me to find out how to use it is through the 'open colab' button. You could consider adding an example to the README. I personally always try to add a documentation website, which is really easily done with e.g. mkdocs or Sphinx. For an example, you could check out my most recent project deptry.

I have recently been working on a project called `deptry`, a command line tool to check for issues in the dependencies of Python projects. It can be used to find obsolete, missing, transitive and misplaced development dependencies. It supports the following types of projects:

- Projects that use Poetry and a corresponding pyproject.toml file

- Projects that use a requirements.txt file according to the pip standards

---

* Documentation: https://fpgmaas.github.io/deptry/

* GitHub repository: https://github.com/fpgmaas/deptry

---

I am quite happy with the project in its current form, but I also realise there is still a lot of room left for improvement. Therefore, I hope some people are willing to give it a try and provide me with feedback. So; if you have a project with a long list of dependencies and a little bit of spare time on your hands, please give it a try and let me know what you think!

If you encounter any issues, find a bug, or have any other form of feedback, please don't hesitate to raise an issue in the GitHub repository, or leave a comment here.

Kind regards,

Florian

P.S. Many thanks to Hirokazu Takaya (https://github.com/lisphilar) for incorporating it in the CI/CD pipeline of his project covid19-sir (https://github.com/lisphilar/covid19-sir). It provided me with very valuable early feedback.

- Projects that use a _requirements.txt_ file according to the [pip](https://pip.pypa.io/en/stable/user_guide/) standards

* [*Documentation*](https://fpgmaas.github.io/deptry/)

There is a visual demo here: https://sites.google.com/view/llm4html/home.

This work is very exciting for a few reasons:

* HTML is an incredibly rich source of visually structured information, with a semi-structured representation. This is as opposed to PDFs, which are usually fed into models with a "flat" representation (words + bounding boxes). Intuitively, this offers the model a more direct way to learn about nested structure, over an almost unlimited source of unsupervised pre-training data.

* Many projects (e.g. Pix2Struct https://arxiv.org/pdf/2210.03347.pdf, also from Google) operate on pixels, which are expensive (both to render and process in the transformer). Operating on HTML directly means smaller, faster, more efficient models.

* (If open sourced) it will be the first (AFAIK) open foundation model for the RPA/automation space (there are several closed projects). They claim they plan to open source the dataset at least, which is very exciting.

I'm particularly excited to extend this and similar (https://arxiv.org/abs/2110.08518) for HTML question answering and web scraping.

Disclaimer: I'm the CEO of Impira, which creates OSS (https://github.com/impira/docquery) and proprietary (http://impira.com/) tools for analyzing business documents. I am not affiliated with this project.

DocQuery (https://github.com/impira/docquery), a project I work on, allows you to do something similar, but search over semantic information in the PDF files (using a large language model that is pre-trained to query business documents).

For example:

  $ docquery scan "What is the due date?" /my/invoices/

docquery – An easy way to extract information from documents