pachyderm
trivy
Our great sponsors
pachyderm | trivy | |
---|---|---|
8 | 82 | |
6,071 | 21,222 | |
0.3% | 3.1% | |
9.8 | 9.7 | |
7 days ago | 3 days ago | |
Go | Go | |
Apache License 2.0 | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
pachyderm
-
Open Source Advent Fun Wraps Up!
20. Pachyderm | Github | tutorial
-
Exploring Open-Source Alternatives to Landing AI for Robust MLOps
Pachyderm specializes in creating compliance-focused pipelines that integrate with enterprise-level storage solutions.
-
Show HN: We scaled Git to support 1 TB repos
There are a couple of other contenders in this space. DVC (https://dvc.org/) seems most similar.
If you're interested in something you can self-host... I work on Pachyderm (https://github.com/pachyderm/pachyderm), which doesn't have a Git-like interface, but also implements data versioning. Our approach de-duplicates between files (even very small files), and our storage algorithm doesn't create objects proportional to O(n) directory nesting depth as Xet appears to. (Xet is very much like Git in that respect.)
The data versioning system enables us to run pipelines based on changes to your data; the pipelines declare what files they read, and that allows us to schedule processing jobs that only reprocess new or changed data, while still giving you a full view of what "would" have happened if all the data had been reprocessed. This, to me, is the key advantage of data versioning; you can save hundreds of thousands of dollars on compute. Being able to undo an oopsie is just icing on the cake.
Xet's system for mounting a remote repo as a filesystem is a good idea. We do that too :)
-
Awesome list of VCs investing in commercial open-source startups
Pachyderm - License prevents competition.
-
Airflow's Problem
I was at Airbnb when we open-sourced Airflow, it was a great solution to the problems we had at the time. It's amazing how many more use cases people have found for it since then. At the time it was pretty focused on solving our problem of orchestrating a largely static DAG of SQL jobs. It could do other stuff even then, but that was mostly what we were using it for. Airflow has become a victim of its success as it's expanded to meet every problem which could ever be considered a data workflow. The flaws and horror stories in the post and comments here definitely resonate with me. Around the time Airflow was opensource I starting working on data-centric approach to workflow management called Pachyderm[0]. By data-centric I mean that it's focused around the data itself, and its storage, versioning, orchestration and lineage. This leads to a system that feels radically different from a job focused system like Airflow. In a data-centric system your spaghetti nest of DAGs is greatly simplified as the data itself is used to describe most of the complexity. The benefit is that data is a lot simpler to reason about, it's not a living thing that needs to run in a certain way, it just exists, and because it's versioned you have strong guarantees about how it can change.
-
One secret tip for first-time OSS contributors. Shh! 🤫 don't tell anyone else
Here is a demo run of lgtm on pachyderm
- Dud: a tool for versioning data alongside source code, written in Go
trivy
-
A Deep Dive Into Terraform Static Code Analysis Tools: Features and Comparisons
Trivy Owner/Maintainer: Aqua Security Age: First released on GitHub on May 7th, 2019 License: Apache License 2.0 backward-compatible with tfsec
- Suas imagens de container não estão seguras!
-
General Docker Troubleshooting, Best Practices & Where to Go From Here
Trivy. A Simple and Comprehensive Vulnerability Scanner for Containers.
-
Distroless images using melange and apko
Using Trivy:
- Friends - needs help choosing solution for SBOM vulnerability
-
An Overview of Kubernetes Security Projects at KubeCon Europe 2023
Trivy is a mature and comprehensive open source tool from Aqua Security that supports scanning multiple sources, from file systems to containers and VMs. Trivy also looks beyond vulnerabilities, to scan licenses, secrets, infrastructure as code misconfiguration, and more.
- Best vulnerability scanner for DevOps
-
About Cloudflare Tunnels
I would suggest to think about the thread model that you are facing so you can have a better mental model of the weak points of your environment. The very very big majority of these attacks will be automated probing for publicly known vulnerabilities or default credentials. That means the maintainers of the software you are running and the channels on which their updates are shipped to you and deployed are very important factors. For software that is not installed from a trusted and well maintained source (e.g. Ubuntus main repository), you want to make extra sure that vulnerabilities are updated. E.g. your deployed docker containers might contain security issues, you can run checks on these with tools like trivy. The same is also true for appliances, in case your router or firewall contains a software vulnerability, how will you be notified and how will the required updates be deployed?
- Security docker app
-
Open source container scanning tool to find vulnerabilities and suggest best practice improvements?
https://github.com/aquasecurity/trivy 17k stars, updated 11 hours ago
What are some alternatives?
snyk - Snyk CLI scans and monitors your projects for security vulnerabilities. [Moved to: https://github.com/snyk/cli]
grype - A vulnerability scanner for container images and filesystems
clair - Vulnerability Static Analysis for Containers
checkov - Prevent cloud misconfigurations and find vulnerabilities during build-time in infrastructure as code, container images and open source packages with Checkov by Bridgecrew.
syft - CLI tool and library for generating a Software Bill of Materials from container images and filesystems
falco - Cloud Native Runtime Security
dockle - Container Image Linter for Security, Helping build the Best-Practice Docker Image, Easy to start
dependency-track - Dependency-Track is an intelligent Component Analysis platform that allows organizations to identify and reduce risk in the software supply chain.
hadolint - Dockerfile linter, validate inline bash, written in Haskell
kics - Find security vulnerabilities, compliance issues, and infrastructure misconfigurations early in the development cycle of your infrastructure-as-code with KICS by Checkmarx.
kube-bench - Checks whether Kubernetes is deployed according to security best practices as defined in the CIS Kubernetes Benchmark
tfsec - Security scanner for your Terraform code