horovod VS petastorm

Compare horovod vs petastorm and see what are their differences.

horovod

Distributed training framework for TensorFlow, Keras, PyTorch, and Apache MXNet. (by horovod)

petastorm

Petastorm library enables single machine or distributed training and evaluation of deep learning models from datasets in Apache Parquet format. It supports ML frameworks such as Tensorflow, Pytorch, and PySpark and can be used from pure Python code. (by uber)
Our great sponsors
  • Nanos - Run Linux Software Faster and Safer than Linux with Unikernels
  • Scout APM - A developer's best friend. Try free for 14-days
  • SaaSHub - Software Alternatives and Reviews
horovod petastorm
3 1
11,898 1,296
0.9% 2.5%
9.3 7.4
2 days ago about 1 month ago
Python Python
GNU General Public License v3.0 or later Apache License 2.0
The number of mentions indicates the total number of mentions that we've tracked plus the number of user suggested alternatives.
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.

horovod

Posts with mentions or reviews of horovod. We have used some of these posts to build our list of alternatives and similar projects.
  • [D] GPU buying recommendation
    1 project | reddit.com/r/MachineLearning | 17 Jul 2021
    If you just want to run tensorflow or pytorch for a Jupyter notebook, setting the environment shouldn't be difficult. I know that AWS has a marketplace of preconfigured images. However, you can go as advanced as setting up a cluster of gpu-equipped nodes to setup Horovod (https://github.com/horovod/horovod) to do distributed machine learning. Yes, there's a learning curve, but you cannot acquire this skillet any other way.
  • SKLean, TensorFlow, etc vs Spark ML?
    1 project | reddit.com/r/apachespark | 12 Feb 2021
    I'm the maintainer for an open source project called Horovod that allows you to distribute deep learning training (e.g., TensorFlow) on platforms like Spark.
  • Cluster machine learning
    1 project | reddit.com/r/HPC | 11 Feb 2021
    You'll want to use horovod to run keras in a distributed system. Then use Slurm to manage the cluster and run the job.

petastorm

Posts with mentions or reviews of petastorm. We have used some of these posts to build our list of alternatives and similar projects.

What are some alternatives?

When comparing horovod and petastorm you can also consider the following projects:

DeepDanbooru - AI based multi-label girl image classification system, implemented by using TensorFlow.

NudeNet - Neural Nets for Nudity Detection and Censoring

onepanel - The open source, end-to-end computer vision platform. Label, build, train, tune, deploy and automate in a unified platform that runs on any cloud and on-premises.

thinc - 🔮 A refreshing functional take on deep learning, compatible with your favorite libraries

d2l-en - Interactive deep learning book with multi-framework code, math, and discussions. Adopted at 300 universities from 55 countries including Stanford, MIT, Harvard, and Cambridge.

AdamP - AdamP: Slowing Down the Slowdown for Momentum Optimizers on Scale-invariant Weights (ICLR 2021)

pytorch-summary - Model summary in PyTorch similar to `model.summary()` in Keras

jina - Cloud-native neural search framework for 𝙖𝙣𝙮 kind of data

best-of-ml-python - 🏆 A ranked list of awesome machine learning Python libraries. Updated weekly.

nni - An open source AutoML toolkit for automate machine learning lifecycle, including feature engineering, neural architecture search, model compression and hyper-parameter tuning.

einops - Deep learning operations reinvented (for pytorch, tensorflow, jax and others)