high-throughput

Open-source projects categorized as high-throughput
Edit details
Language: + Python + Go + C++ + Java
Topics: Deep Learning Machine Learning low-latency offloading fastest
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com
featured

Top 6 high-throughput Open-Source Projects

  • FlexGen

    Running large language models on a single GPU for throughput-oriented scenarios.

    • Project mention: Run 70B LLM Inference on a Single 4GB GPU with This New Technique | news.ycombinator.com | 2023-12-03

  • ZenQ

    A thread-safe queue faster and more resource efficient than golang's native channels

  • InfluxDB

    Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

    InfluxDB logo

  • warp-drive

    Extremely Fast End-to-End Deep Multi-Agent Reinforcement Learning Framework on a GPU (JMLR 2022)

  • LruClockCache

    A low-latency LRU approximation cache in C++ using CLOCK second-chance algorithm. Multi level cache too. Up to 2.5 billion lookups per second.

  • mangoo I/O

    An Intuitive, Lightweight, High Performance Full Stack Java Web Framework.

  • zarp

    The Zavolab Automated RNA-seq Pipeline (by zavolanlab)

NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020).

high-throughput related posts

  • Run 70B LLM Inference on a Single 4GB GPU with This New Technique

    3 projects | news.ycombinator.com | 3 Dec 2023

  • Colorful Custom RTX 4060 Ti GPU Clocks Outed, 8 GB VRAM Confirmed

    1 project | /r/hardware | 17 Apr 2023

  • FlexGen: Running large language models on a single GPU

    1 project | /r/hypeurls | 26 Mar 2023

  • FlexGen: Running large language models on a single GPU

    1 project | /r/patient_hackernews | 26 Mar 2023

  • FlexGen: Running large language models on a single GPU

    1 project | /r/hackernews | 26 Mar 2023

  • FlexGen: Running large language models on a single GPU

    4 projects | news.ycombinator.com | 25 Mar 2023

  • Could this new flexgen be used in place of GPTq? or is this different?

    1 project | /r/Oobabooga | 18 Mar 2023
  • A note from our sponsor - SaaSHub
    www.saashub.com | 22 May 2024
    SaaSHub helps you find the best software and product alternatives Learn more →

Index

What are some of the best open-source high-throughput projects? This list will help you:

Project Stars
1 FlexGen 9,035
2 ZenQ 636
3 warp-drive 436
4 LruClockCache 59
5 mangoo I/O 55
6 zarp 29

Sponsored
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com