pip-audit
dvc
pip-audit | dvc | |
---|---|---|
22 | 109 | |
920 | 13,166 | |
1.4% | 1.0% | |
8.8 | 9.6 | |
10 days ago | 4 days ago | |
Python | Python | |
Apache License 2.0 | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
pip-audit
-
Smooth Packaging: Flowing from Source to PyPi with GitLab Pipelines
Next up is making sure, none of the dependencies used throughout the project brings with it any already identified security issue. The makefile target audit, invokes the handy tool pip-audit.
-
Show HN: One makefile to rule them all
Here is my "one true" Makefile for Python projects[1]. The skeleton gets tweaked slightly each time, but it's served me well for 4+ years.
[1]: https://github.com/pypa/pip-audit/blob/main/Makefile
-
Pyscan: A command-line tool to detect security issues in your python dependencies.
Why use this over the established https://pypi.org/project/pip-audit/ ?
-
How Attackers Can Sneakily Slip Malware Packages Into Poetry.lock Files
https://pypi.org/project/pip-audit/ details usage and the GitHub Action install.
- How to improve Python packaging, or why 14 tools are at least 12 too many
-
Underappreciated Challenges with Python Packaging
If it's pure Python, the only packaging file you need is `pyproject.toml`. You can fill that file with packaging metadata per PEP 518 and PEP 621, including using modern build tooling like flit[1] for the build backend and build[2] for the frontend.
With that, you entire package build (for all distribution types) should be reducible to `python -m build`. Here's an example of a full project doing everything with just `pyproject.toml`[3] (FD: my project).
[1]: https://github.com/pypa/flit
[2]: https://github.com/pypa/build
[3]: https://github.com/pypa/pip-audit
-
Auditing your python environment
- repo: https://github.com/trailofbits/pip-audit rev: v2.4.3 hooks: - id: pip-audit args: [ "-r", "requirements.txt" ] ci: # Leave pip-audit to only run locally and not in CI # pre-commit.ci does not allow network calls skip: [ pip-audit ]
-
How to create a Python package in 2022
This is really nicely written; kudos to the author for compiling a great deal of information in a readable format.
If I can be forgiven one nitpick: Poetry does not use a PEP 518-style[1] build configuration by default, which means that its use of `pyproject.toml` is slightly out of pace with the rest of the Python packaging ecosystem. That isn't to say that it isn't excellent, because it is! But you the standards have come a long way, and you can now use `pyproject.toml` with any build backend as long as you use the standard metadata.
By way of example, here's a project that's completely PEP 517 and PEP 518 compatible without needing a setup.py or setup.cfg[2]. Everything goes through pyproject.toml.
[1]: https://peps.python.org/pep-0518/
[2]: https://github.com/trailofbits/pip-audit/blob/main/pyproject...
-
I think the CTX package on PyPI has been hacked!
Checking could be done if something like this eventually shows up in safety or pip-audit.
-
Open-source way to scan dependencies for CVEs?
Something like python's pip-audit. For commercial solutions I know there's Snyk and Jfrog we can always purchase, but I'm interested to see if there's an open-source tool that can do this.
dvc
-
My Favorite DevTools to Build AI/ML Applications!
Collaboration and version control are crucial in AI/ML development projects due to the iterative nature of model development and the need for reproducibility. GitHub is the leading platform for source code management, allowing teams to collaborate on code, track issues, and manage project milestones. DVC (Data Version Control) complements Git by handling large data files, data sets, and machine learning models that Git can't manage effectively, enabling version control for the data and model files used in AI projects.
-
Why bad scientific code beats code following "best practices"
What you’re describing sounds like DVC (at a higher-ish—80%-solution level).
https://dvc.org/
See pachyderm too.
-
First 15 Open Source Advent projects
10. DVC by Iterative | Github | tutorial
-
Exploring Open-Source Alternatives to Landing AI for Robust MLOps
Platforms such as MLflow monitor the development stages of machine learning models. In parallel, Data Version Control (DVC) brings version control system-like functions to the realm of data sets and models.
- ML Experiments Management with Git
-
Git Version Controlled Datasets in S3
I was using DVC (https://dvc.org/) for some time to help solve this but it was getting hard to manage the storage connections and I would run into cache issues a lot, but this solves it using git-lfs itself.
- Ask HN: How do your ML teams version datasets and models?
-
Exploring MLOps Tools and Frameworks: Enhancing Machine Learning Operations
DVC (Data Version Control):
- Evaluate and Track Your LLM Experiments: Introducing TruLens for LLMs
-
[D] Is there a tool to keep track of my ML experiments?
I have been using DVC and MLflow since then DVC had only data tracking and MLflow only model tracking. I can say both are awesome now and maybe the only factor I would like to mention is that IMO, MLflow is a bit harder to learn while DVC is just a git practically.
What are some alternatives?
ochrona-cli - A command line tool for detecting vulnerabilities in Python dependencies and doing safe package installs
MLflow - Open source platform for the machine learning lifecycle
git-hooks.nix - Seamless integration of https://pre-commit.com git hooks with Nix.
lakeFS - lakeFS - Data version control for your data lake | Git for data
npm-esbuild-audit
Activeloop Hub - Data Lake for Deep Learning. Build, manage, query, version, & visualize datasets. Stream data real-time to PyTorch/TensorFlow. https://activeloop.ai [Moved to: https://github.com/activeloopai/deeplake]
setup-dvc - DVC GitHub action
delta - An open-source storage framework that enables building a Lakehouse architecture with compute engines including Spark, PrestoDB, Flink, Trino, and Hive and APIs
aura - Python source code auditing and static analysis on a large scale
ploomber - The fastest ⚡️ way to build data pipelines. Develop iteratively, deploy anywhere. ☁️
tox-poetry-installer - A plugin for Tox that lets you install test environment dependencies from the Poetry lockfile
aim - Aim 💫 — An easy-to-use & supercharged open-source experiment tracker.