leptonai
examples
| leptonai | examples | |
|---|---|---|
| 2 | 1 | |
| 2,469 | 133 | |
| 5.7% | 4.5% | |
| 9.7 | 8.4 | |
| 3 days ago | about 2 months ago | |
| Python | Jupyter Notebook | |
| Apache License 2.0 | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
leptonai
-
Show HN: Running LLMs in one line of Python without Docker
Hello Hacker News! We're Yangqing, Xiang and JJ from lepton.ai. We are building a platform to run any AI models as easy as writing local code, and to get your favorite models in minutes. It's like container for AI, but without the hassle of actually building a docker image.
We built and contributed to some of the world's most popular AI software - PyTorch 1.0, ONNX, Caffe, etcd, Kubernetes, etc. We also managed hundreds of thousands of computers in our previous jobs. And we found that the AI software stack is usually unnecessarily complex - and we want to change that.
Imagine if you are a developer who sees a good model on github, or HuggingFace. To make it a production ready service, the current solution usually requires you to build a docker image. But think about it - I have a few python code and a few python dependencies. That sounds like a huge overhead, right?
lepton.ai is really a pythonic way to free you from such difficulties. You write a simple python scaffold around your PyTorch / TensorFlow code, and lepton launches it as a full-fledged service callable via python, javascript, or any language that understands OpenAPI. We use containers under the hood, but you don't need to worry about all the infrastructure nuts and bolts.
We have made the python library open-source at https://github.com/leptonai/leptonai/. With it, launching a common HuggingFace model is as simple as a one liner. For example, if you have a GPU, Stable Diffusion XL is as simple as:
```
- Lepton: An open-source library (Apache 2.0) for scaling model inference
examples
-
Show HN: Running LLMs in one line of Python without Docker
It's not only about "building a docker" but also maintaining multiple models, multiple environments and a lot of users. Imagine there is a group of engineers each needing to deploy their own models: one needs tensorflow 1.x, one needs tensorflow 2.x, one needs pytorch and one needs a very strange combination of dependencies. Trust me, things get complex very easily:
https://github.com/leptonai/examples/blob/main/advanced/whis...
I definitely agree that for a fixed use case, building a docker once and for all is probably the simplest and best approach. However, it quickly gets more complex and out of hand.
Also the basic plan is free for independent developers - isn't it worth it? :)
What are some alternatives?
Pytorch - Tensors and Dynamic neural networks in Python with strong GPU acceleration
chainer - A flexible framework of neural networks for deep learning
ivy - The Unified Machine Learning Framework [Moved to: https://github.com/unifyai/ivy]
ivy - The Unified AI Framework
server - The Triton Inference Server provides an optimized cloud and edge inferencing solution.
ImageAI - A python library built to empower developers to build applications and systems with self-contained Computer Vision capabilities
caer - High-performance Vision library in Python. Scale your research, not boilerplate.