transformer-deploy

Efficient, scalable and enterprise-grade CPU/GPU inference server for 🤗 Hugging Face transformer models 🚀 (by ELS-RD)

Transformer-deploy Alternatives

Similar projects and alternatives to transformer-deploy

  1. Nutrient

    Nutrient - The #1 PDF SDK Library. Bad PDFs = bad UX. Slow load times, broken annotations, clunky UX frustrates users. Nutrient’s PDF SDKs gives seamless document experiences, fast rendering, annotations, real-time collaboration, 100+ features. Used by 10K+ devs, serving ~half a billion users worldwide. Explore the SDK for free.

    Nutrient logo
  2. diffusers

    🤗 Diffusers: State-of-the-art diffusion models for image, video, and audio generation in PyTorch and FLAX.

  3. onnxruntime

    ONNX Runtime: cross-platform, high performance ML inferencing and training accelerator

  4. server

    The Triton Inference Server provides an optimized cloud and edge inferencing solution. (by triton-inference-server)

  5. TensorRT

    NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference on NVIDIA GPUs. This repository contains the open source components of TensorRT.

  6. flash-attention

    Fast and memory-efficient exact attention

  7. deepsparse

    Sparsity-aware deep learning inference runtime for CPUs

  8. CodeRabbit

    CodeRabbit: AI Code Reviews for Developers. Revolutionize your code reviews with AI. CodeRabbit offers PR summaries, code walkthroughs, 1-click suggestions, and AST-based analysis. Boost productivity and code quality across all major languages with each PR.

    CodeRabbit logo
  9. BentoML

    The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!

  10. FasterTransformer

    Transformer related optimization, including BERT, GPT

  11. optimum

    🚀 Accelerate inference and training of 🤗 Transformers, Diffusers, TIMM and Sentence Transformers with easy to use hardware optimization tools

  12. TensorRT

    PyTorch/TorchScript/FX compiler for NVIDIA GPUs using TensorRT (by pytorch)

  13. fastT5

    âš¡ boost inference speed of T5 models by 5x & reduce the model size by 3x.

  14. serve

    Serve, optimize and scale PyTorch models in production (by pytorch)

  15. mmrazor

    OpenMMLab Model Compression Toolbox and Benchmark.

  16. kernl

    8 transformer-deploy VS kernl

    Kernl lets you run PyTorch transformer models several times faster on GPU with a single line of code, and is designed to be easily hackable.

  17. torch2trt

    An easy to use PyTorch to TensorRT converter

  18. openai-whisper-cpu

    Improving transcription performance of OpenAI Whisper for CPU based deployment

  19. parallelformers

    Parallelformers: An Efficient Model Parallelization Toolkit for Deployment

  20. OpenSeeFace

    Robust realtime face and facial landmark tracking on CPU with Unity integration

  21. sparsednn

    Fast sparse deep learning on CPUs

  22. SaaSHub

    SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

    SaaSHub logo
NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a better transformer-deploy alternative or higher similarity.

transformer-deploy discussion

Log in or Post with

transformer-deploy reviews and mentions

Posts with mentions or reviews of transformer-deploy. We have used some of these posts to build our list of alternatives and similar projects. The last one was on 2022-10-28.

Stats

Basic transformer-deploy repo stats
8
1,673
0.0
4 months ago

Sponsored
Nutrient - The #1 PDF SDK Library
Bad PDFs = bad UX. Slow load times, broken annotations, clunky UX frustrates users. Nutrient’s PDF SDKs gives seamless document experiences, fast rendering, annotations, real-time collaboration, 100+ features. Used by 10K+ devs, serving ~half a billion users worldwide. Explore the SDK for free.
nutrient.io

Did you know that Python is
the 2nd most popular programming language
based on number of references?