Module to Automatically maximize the utilization of GPU resources in a Kubernetes cluster through real-time dynamic partitioning and elastic quotas - Effortless optimization at its finest! (by nebuly-ai)

Nos Alternatives

Similar projects and alternatives to nos based on common topics and language

  • nebuly

    nos VS nebuly

    The next-generation platform to monitor and optimize your AI costs in one place 🚀

  • k8s-device-plugin

    NVIDIA device plugin for Kubernetes

  • InfluxDB

    Access the most powerful time series database as a service. Ingest, store, & analyze all types of time series data in a fully-managed, purpose-built database. Keep data forever with low-cost storage and superior data compression.

  • pytorch-accelerated

    A lightweight library designed to accelerate the process of training PyTorch models by providing a minimal, but extensible training loop which is flexible enough to handle the majority of use cases, and capable of utilizing different hardware options with no code changes required. Docs:

  • olivia

    nos VS olivia

    💁‍♀️Your new best friend powered by an artificial neural network

  • Activeloop Hub

    nos VS Activeloop Hub

    Data Lake for Deep Learning. Build, manage, query, version, & visualize datasets. Stream data real-time to PyTorch/TensorFlow. [Moved to:] (by activeloopai)

  • lightning

    nos VS lightning

    Deep learning framework to train, deploy, and ship AI products Lightning fast.

  • pytorch-lightning

    nos VS pytorch-lightning

    Build high-performance AI models with PyTorch Lightning (organized PyTorch). Deploy models with Lightning Apps (organized Python to build end-to-end ML systems). [Moved to:]

  • SonarQube

    Static code analysis for 29 languages.. Your projects are multi-language. So is SonarQube analysis. Find Bugs, Vulnerabilities, Security Hotspots, and Code Smells so you can release quality code every time. Get started analyzing your projects today for free.

NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a better nos alternative or higher similarity.

nos reviews and mentions

Posts with mentions or reviews of nos. We have used some of these posts to build our list of alternatives and similar projects. The last one was on 2023-03-01.
  • Plug and play modules to optimize the performances of your AI systems
    3 projects | | 1 Mar 2023
    Some of the available modules include:

    Speedster: Automatically apply the best set of SOTA optimization techniques to achieve the maximum inference speed-up on your hardware.

    Nos: Automatically maximize the utilization of GPU resources in a Kubernetes cluster through real-time dynamic partitioning and elastic quotas.

    ChatLLaMA: Build faster and cheaper ChatGPT-like training process based on LLaMA architectures.

    OpenAlphaTensor: Increase the computational performances of an AI model with custom-generated matrix multiplication algorithm fine-tuned for your specific hardware.

    Forward-Forward: The Forward Forward algorithm is a method for training deep neural networks that replaces the backpropagation forward and backward passes with two forward passes.

  • Nos – Open-Source to Maximize GPU Utilization in Kubernetes
    3 projects | | 9 Feb 2023
    Hi HN! I’m Michele Zanotti and today I’m releasing nos, an open-source module to efficiently run GPU workloads on Kubernetes!

    Nos is meant to increase GPU utilization and cut down infrastructure and operational costs providing 2 main features:

    1. Dynamic GPU Partitioning: you can think of this as a cluster autoscaler for GPUs. Instead of scaling up the number of nodes and GPUs, it dynamically partitions them into smaller “GPU slices”. This ensures that each workload only uses the GPU resources it actually needs, resulting in spare GPU capacity that could be used for other workloads. To partition GPUs, nos leverages Nvidia's MPS and MIG [1,2], finally making them dynamic.

    2. Elastic Resource Quota management: it allows to increase the number of Pods running on the cluster by allowing teams (namespaces) to borrow quotas of reserved resources from other teams as long as they are not using them.

    Let me know your thoughts on the project in the comments. And don't forget to leave a star on GitHub if you like the project :)

    Nos addresses some key challenges of Kubernetes tied to the fact that Kubernetes was not designed to support GPU and AI / machine learning workloads. In Kubernetes, GPUs are managed with [3] Nvidia k8s Device Plugin, which has a few major downsides. First, it requires the allocation of an integer number of GPUs per workload, not allowing workloads to request only fractions of GPU. Second, when enabling GPU shared access either with time-slicing or MIG, the device plugin advertises to Kubernetes a fixed set of GPU resources that do not dynamically adapt to the requests of the Pods at each time.

    This often leads to both underutilized GPUs and pending Pods, and/or the cluster admin having to spend a lot of time looking for workarounds to make the best use of GPUs.

    For example, consider a company with a k8s cluster with 20 GPUs, where 3 of these GPUs have been reserved for the data science team using Resource Quota objects. In most cases, the workloads of data scientists (notebooks, scripts, etc.) require much less memory/compute resources than those of an entire GPU, yet Kubernetes will force each container to consume an entire GPU. Also, if the team once needs to run a heavy workload, it may want to use as many resources as possible. However, the Resource Quota over their namespace would constrain the team to use at most the 3 GPUs reserved for them, even if the company cluster may be full of unused GPUs!

    Instead, with nos the data science team would use nos Dynamic GPU Partitioning to request GPU slices so that many workloads can share the same GPU. Also, Elastic Resource Quotas would allow the team to consume more than the 3 reserved GPUs, borrowing quotas from other teams that are not using them. To recap, the team would be able to launch more Pods and the company would likely need fewer nodes. All this with minimal effort required by the cluster admin, who only has to set up nos.

    Let me know what you think of nos, feedback would be very helpful! :) And please leave a star on GitHub if you like this opensource

    Here are some other links that may be useful

    - Tutorial on how to use Dynamic GPU Partitioning with Nvidia MIG

    3 projects | | 9 Feb 2023
  • Show HN: Nos – Open-Source to Maximize GPU Utilization in Kubernetes
    2 projects | | 30 Jan 2023
  • A note from our sponsor - #<SponsorshipServiceOld:0x00007f0920bc6850> | 10 Jun 2023
    SaaSHub helps you find the best software and product alternatives Learn more →


Basic nos repo stats
21 days ago

nebuly-ai/nos is an open source project licensed under Apache License 2.0 which is an OSI approved license.

The primary programming language of nos is Go.

Popular Comparisons

SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives