Is there any open source app to load a model and expose API like OpenAI?

• llm-api

  2 143 6.6 Python

  Run any Large Language Model behind a unified API

1b5d/llm-api: Run any Large Language Model behind a unified API (github.com)

• litellm

  28 8,225 10.0 Python

  Call all LLM APIs using the OpenAI format. Use Bedrock, Azure, OpenAI, Cohere, Anthropic, Ollama, Sagemaker, HuggingFace, Replicate (100+ LLMs)

  

I use this with ollama and works perfectly https://github.com/BerriAI/litellm

• InfluxDB

  www.influxdata.com sponsored

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• LocalAI

  82 19,593 9.9 C++

  :robot: The free, Open Source OpenAI alternative. Self-hosted, community-driven and local-first. Drop-in replacement for OpenAI running on consumer-grade hardware. No GPU required. Runs gguf, transformers, diffusers and many more models architectures. It allows to generate Text, Audio, Video, Images. Also with voice cloning capabilities.

  

• text-generation-inference

  29 7,881 9.6 Python

  Large Language Model Text Generation Inference

  

• server

  24 7,314 9.5 Python

  The Triton Inference Server provides an optimized cloud and edge inferencing solution. (by triton-inference-server)

  

• WorkOS

  workos.com sponsored

  The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

  

Suggest a related project