Top 7 Python serving Projects

serving

• Add a project

• Ray

  45 33,295 10.0 Python

  Ray is a unified framework for scaling AI and Python applications. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.

  

  Project mention: Amazon's Exabyte-Scale Migration from Apache Spark to Ray on Amazon EC2 | news.ycombinator.com | 2024-07-29

  Yeah, mmap, I think this is the relevant line [1].

  Fun fact, very early on, we used to create one mmapped file per serialized object, but that very quickly broke down.

  Then we switched to mmapping one large file at the start and storing all of the serialized objects in that file. But then as objects get allocated and deallocated, you need to manage the memory inside of that mmapped file, and we just repurposed a malloc implementation to handle that.

  [1] https://github.com/ray-project/ray/blob/21202f6ddc3ceaf74fbc...

• SaaSHub

  www.saashub.com featured

  SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

  

• evadb

  27 2,630 9.3 Python

  Database system for AI-powered apps

  

• LitServe

  4 2,203 9.8 Python

  Lightning-fast serving engine for AI models. Flexible. Easy. Enterprise-scale.

  

  Project mention: AIM Weekly for 23 September 2024 | dev.to | 2024-09-23

  📎 Scaling Databases for GenAI 🤖 Streaming Vectors Webinar 📊 SQL, NoSQL, Vectors 📱 Super new model from MS - GRIN MoE 🛼 Salesforce Keynote 📢 RDBMS vs Vector DB 🐈‍⬛ DBTA Top 75 🌐 LitServe 📊 Langchain with Filtering 🖥️ PDF Extract Kit 👽 LLM Testing 🖥️ Easy Milvus Schema Generation 🌐Uber's Query GPT

• MetaSpore

  11 642 3.1 Python

  A unified end-to-end machine intelligence platform

  

• pinferencia

  21 561 0.0 Python

  Python + Inference - Model Deployment library in Python. Simplest model inference server ever.

  

• fastDeploy

  1 96 7.9 Python

  Deploy DL/ ML inference pipelines with minimal extra code. (by notAI-tech)

  

  Project mention: Ask HN: Is there any software you only made for your own use but nobody else? | news.ycombinator.com | 2024-07-04

  I made https://github.com/notAI-tech/fastDeploy for ease of use of deploying Deep learning models.

  Except me and the team at my company nobody uses it.

  After I made it lot of solutions came in this space, but IMO this is the easiest one to use still.

• sdk-python

  1 24 4.5 Python

  Python library for Modzy Machine Learning Operations (MLOps) Platform (by modzy)

  

Python serving related posts

• A full-stack Python model serving library

  1 project | news.ycombinator.com | 28 Aug 2024

• LitServe: Fast serving engine for AI models

  1 project | news.ycombinator.com | 27 Aug 2024

• Aviary: Compare Open Source LLMs for cost, latency and quality

  1 project | news.ycombinator.com | 1 Jun 2023

• [N] Aviary: Comparing Open Source LLMs for cost, latency and quality

  1 project | /r/MachineLearning | 1 Jun 2023

• Anyscale's Aviary is a dashboard for evaluating Open Source LLMs

  1 project | news.ycombinator.com | 31 May 2023

• Popular Machine Learning Deployment Tools

  4 projects | dev.to | 16 Apr 2022
• A note from our sponsor - SaaSHub
  www.saashub.com | 6 Oct 2024

  SaaSHub helps you find the best software and product alternatives Learn more →

Index

Sponsored

SaaSHub - Software Alternatives and Reviews

SaaSHub helps you find the best software and product alternatives

www.saashub.com

Do not miss the trending Python projects with our weekly report!