runbooks
Awesome-LLM-Productization
| runbooks | Awesome-LLM-Productization | |
|---|---|---|
| 4 | 2 | |
| 156 | 19 | |
| 3.8% | - | |
| 8.8 | 4.6 | |
| 6 months ago | 7 months ago | |
| Go | ||
| GNU General Public License v3.0 or later | GNU General Public License v3.0 or later |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
runbooks
- Deploy and Fine-tune large language models on k8s - Trying this out this weekend
-
Show HN: Kubectl Notebook
Thanks for the feedback! All resources are namespaced right now, except there is an issue the notebook plugin where namespaces are indeed broken: https://github.com/substratusai/substratus/issues/193 Will get that fixed very soon.
- Deploy and fine-tune large language models on K8s
Awesome-LLM-Productization
-
Git repo focusing on productizing LLMs/AI
There are a bunch of open source projects or commercial projects productising LLMs, but there are still challenges in, e.g., latency, cost reduction, fine-tuning, data preparation, monitoring to name a few.
This repo monitors projects or packages that can help you speed up the adoption, with boilerplate, E2E backend and real-world use cases as its goal.
Please feel free to open issues and more contents will be coming soon.
https://github.com/oscinis-com/Awesome-LLM-Productization/
What are some alternatives?
flyte - Scalable and flexible workflow orchestration platform that seamlessly unifies data, ML and analytics stacks.
starwhale - an MLOps/LLMOps platform
kubernetes-operator-roiergasias - 'Roiergasias' kubernetes operator is meant to address a fundamental requirement of any data science / machine learning project running their pipelines on Kubernetes - which is to quickly provision a declarative data pipeline (on demand) for their various project needs using simple kubectl commands. Basically, implementing the concept of No Ops. The fundamental principle is to utilise best of docker, kubernetes and programming language features to run a workflow with minimal workflow definition syntax. It is a Go based workflow running on command line or Kubernetes with the help of a custom operator for a quick and automated data pipeline for your machine learning projects (a flavor of MLOps).
vllm - A high-throughput and memory-efficient inference and serving engine for LLMs
openmodelz - One-click machine learning deployment (LLM, text-to-image and so on) at scale on any cluster (GCP, AWS, Lambda labs, your home lab, or even a single machine).
app - BitGPT it's your personal AI in your pocket
awesome-dolly - A curated list of Databricks' Dolly implementations, documentation, and use cases
awesome-ai-safety - 📚 A curated list of papers & technical articles on AI Quality & Safety
aici - AICI: Prompts as (Wasm) Programs