onnx
dvc
onnx | dvc | |
---|---|---|
41 | 112 | |
17,077 | 13,311 | |
1.3% | 1.5% | |
9.4 | 9.6 | |
9 days ago | 4 days ago | |
Python | Python | |
Apache License 2.0 | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
onnx
-
Using Google Magika to build an AI-powered file type detector
To perform fast inference at runtime, Magika uses the cross-platform Open Neural Network Exchange (ONNX) runtime. ONNX provides a method to optimize, accelerate, and deploy models built using any of the popular frameworks consistently, even across different hardware platforms or instruction set architectures.
-
Nvidia and Salesforce double down on AI startup Cohere in $450M round
Right; but you can't cross-compile everything. This is really common in AI libraries, especially multi-target projects like ONNX: https://onnx.ai/
The math probably adds up in Google's favor with the TPUs, even if they end up being less efficient and slower per-unit than Nvidia hardware. They don't need to pay for the margins, and they can run them 24/7 for their intended purpose. The previous-generation TPUs can't be reused or resold for other purposes though, and if/when AI blows over as a trend you probably can't easily start mining crypto or doing HPC calculations like an Nvidia cluster would.
-
HuggingFace hacked â Space secrets leak disclosure
> I had assumed model files were big matrices of numbers and some metadata perhaps
ONNX [1] is more or less this, but the challenge you immediately run into is models with custom layers/operators with their own inference logic - you either have to implement those operators in terms of the supported ops (not necessarily practical or viable) or provide the implementation of the operator to the runtime, putting you back at square one.
[1] https://onnx.ai/
- Onyx, a new programming language powered by WebAssembly
-
From Lab to Live: Implementing Open-Source AI Models for Real-Time Unsupervised Anomaly Detection in Images
Once your model has been trained and validated using Anomalib, the next step is to prepare it for real-time implementation. This is where ONNX (Open Neural Network Exchange) or OpenVINO (Open Visual Inference and Neural network Optimization) comes into play.
-
Object detection with ONNX, Pipeless and a YOLO model
ONNX is an open format from the Linux Foundation to represent machine learning models. It is becoming extensively adopted by the Machine Learning community and is compatible with most of the machine learning frameworks like PyTorch, TensorFlow, etc. Converting a model between any of those formats and ONNX is really simple and can be done in most cases with a single command.
-
38TB of data accidentally exposed by Microsoft AI researchers
ONNX[0], model-as-protosbufs, continuing to gain adoption will hopefully solve this issue.
[0] https://github.com/onnx/onnx
-
Redditâs LLM text model for Ads Safety
Running inference for large models on CPU is not a new problem and fortunately there has been great development in many different optimization frameworks for speeding up matrix and tensor computations on CPU. We explored multiple optimization frameworks and methods to improve latency, namely TorchScript, BetterTransformer and ONNX.
-
Operationalize TensorFlow Models With ML.NET
ONNX is a format for representing machine learning models in a portable way. Additionally, ONNX models can be easily optimized and thus become smaller and faster.
-
Onnx Runtime: âCross-Platform Accelerated Machine Learningâ
I would say onnx.ai [0] provides more information about ONNX for those who arenât working with ML/DL.
[0] https://onnx.ai
dvc
-
10 Open Source Tools for Building MLOps Pipelines
As Git helps you with code versions and the ability to roll back to previous versions for code repositories, DVC has built-in support for tracking your data and model. This helps machine learning teams reproduce the experiments run by your fellows and facilitates collaboration. DVC is based on the principles of Git and is easy to learn since the commands are similar to those of Git. Other benefits of using DVC include:
-
A step-by-step guide to building an MLOps pipeline
The meta-data and model artifacts from experiment tracking can contain large amounts of data, such as the training model files, data files, metrics and logs, visualizations, configuration files, checkpoints, etc. In cases where the experiment tool doesn't support data storage, an alternative option is to track the training and validation data versions per experiment. They use remote data storage systems such as S3 buckets, MINIO, Google Cloud Storage, etc., or data versioning tools like data version control (DVC) or Git LFS (Large File Storage) to version and persist the data. These options facilitate collaboration but have artifact-model traceability, storage costs, and data privacy implications.
-
AI Strategy Guide: How to Scale AI Across Your Business
Level 1 of MLOps is when you've put each lifecycle stage and their intefaces in an automated pipeline. The pipeline could be a python or bash script, or it could be a directed acyclic graph run by some orchestration framework like Airflow, dagster or one of the cloud-provider offerings. AI- or data-specific platforms like MLflow, ClearML and dvc also feature pipeline capabilities.
-
My Favorite DevTools to Build AI/ML Applications!
Collaboration and version control are crucial in AI/ML development projects due to the iterative nature of model development and the need for reproducibility. GitHub is the leading platform for source code management, allowing teams to collaborate on code, track issues, and manage project milestones. DVC (Data Version Control) complements Git by handling large data files, data sets, and machine learning models that Git can't manage effectively, enabling version control for the data and model files used in AI projects.
-
Why bad scientific code beats code following "best practices"
What youâre describing sounds like DVC (at a higher-ishâ80%-solution level).
https://dvc.org/
See pachyderm too.
-
First 15 Open Source Advent projects
10. DVC by Iterative | Github | tutorial
-
Exploring Open-Source Alternatives to Landing AI for Robust MLOps
Platforms such as MLflow monitor the development stages of machine learning models. In parallel, Data Version Control (DVC) brings version control system-like functions to the realm of data sets and models.
- ML Experiments Management with Git
-
Git Version Controlled Datasets in S3
I was using DVC (https://dvc.org/) for some time to help solve this but it was getting hard to manage the storage connections and I would run into cache issues a lot, but this solves it using git-lfs itself.
- Ask HN: How do your ML teams version datasets and models?
What are some alternatives?
onnxruntime - ONNX Runtime: cross-platform, high performance ML inferencing and training accelerator
MLflow - Open source platform for the machine learning lifecycle
stable-diffusion-webui - Stable Diffusion web UI
lakeFS - lakeFS - Data version control for your data lake | Git for data
stable-diffusion-webui - Stable Diffusion web UI [Moved to: https://github.com/Sygil-Dev/sygil-webui]
Activeloop Hub - Data Lake for Deep Learning. Build, manage, query, version, & visualize datasets. Stream data real-time to PyTorch/TensorFlow. https://activeloop.ai [Moved to: https://github.com/activeloopai/deeplake]
sentence-transformers - Multilingual Sentence & Image Embeddings with BERT
delta - An open-source storage framework that enables building a Lakehouse architecture with compute engines including Spark, PrestoDB, Flink, Trino, and Hive and APIs
stable-diffusion - A latent text-to-image diffusion model
ploomber - The fastest âĄď¸ way to build data pipelines. Develop iteratively, deploy anywhere. âď¸
stable-diffusion-webui - Stable Diffusion web UI [Moved to: https://github.com/sd-webui/stable-diffusion-webui]
aim - Aim đŤ â An easy-to-use & supercharged open-source experiment tracker.