CTranslate2 Alternatives

CTranslate2 reviews and mentions

• Adding 70-language translation to an image API without paying per word
  1 project | dev.to | 9 Jun 2026

  Runtime: CTranslate2 with int8 quantization. This is the key piece. It shrinks the model to ~1.3 GB and runs CPU inference fast. Do not run raw transformers on CPU for this.

• Brood War Korean Translations
  2 projects | news.ycombinator.com | 17 Jan 2025

  Thanks for the added context on the builds! As "foreign" BW player and fellow speech processing researcher, I agree shallow contextual biasing should help. While not difficult to implement, most generally available ASR solutions don't make it easy to use. There's a PR in ctranslate2 implementing the same feature so that it could be exposed in faster-whisper: https://github.com/OpenNMT/CTranslate2/pull/1789

• Creando Subtítulos Automáticos para Vídeos con Python, Faster-Whisper, FFmpeg, Streamlit, Pillow
  7 projects | dev.to | 29 Apr 2024
• Distil-Whisper: distilled version of Whisper that is 6 times faster, 49% smaller
  14 projects | news.ycombinator.com | 31 Oct 2023

  Just a point of clarification - faster-whisper references it but ctranslate2[0] is what's really doing the magic here.

  Ctranslate2 is a sleeper powerhouse project that enables a lot. They should be up front and center and get the credit they deserve.

  [0] - https://github.com/OpenNMT/CTranslate2

• A Raspberry Pi 5 is better than two Pi 4S
  3 projects | news.ycombinator.com | 8 Oct 2023

  We'd love to move beyond Nvidia.

  The issue (among others) is we achieve the speech recognition performance we do largely thanks to ctranslate2[0]. They've gone on the record saying that they essentially have no interest in ROCm[1].

  Of course with open source anything is possible but we see this as being one of several fundamental issues in supporting AMD GPGPU hardware.

  [0] - https://github.com/OpenNMT/CTranslate2

  [1] - https://github.com/OpenNMT/CTranslate2/issues/1072

• AMD May Get Across the CUDA Moat
  8 projects | news.ycombinator.com | 6 Oct 2023
• StreamingLLM: Efficient streaming technique enable infinite sequence lengths
  2 projects | news.ycombinator.com | 3 Oct 2023

  Etc.

  Now, what this allows you to do is reuse the attention computed from the previous turns (since the prefix is the same).

  In practice, people often have a system prompt before the conversation history, which (as far a I can tell) makes this technique not applicable (the input prefix will change as soon as the conversation history is long enough that we need to start dropping the oldest turns).

  In such case, what you could do is to cache at least the system prompt. This is also possible with https://github.com/OpenNMT/CTranslate2/blob/2203ad5c8baf878a...

• Faster Whisper Transcription with CTranslate2
  5 projects | news.ycombinator.com | 20 Jul 2023

  The original Whisper implementation from OpenAI uses the PyTorch deep learning framework. On the other hand, faster-whisper is implemented using CTranslate2 [1] which is a custom inference engine for Transformer models. So basically it is running the same model but using another backend, which is specifically optimized for inference workloads.

  [1] https://github.com/OpenNMT/CTranslate2

• Explore large language models on any computer with 512MB of RAM
  4 projects | /r/LocalLLaMA | 17 Jun 2023

  FLAN-T5 models generally perform well for their size, but they are encode-decoder models, and they aren't as widely supported for efficient inference. I wanted students to be able to run everything locally on CPU, so I was ideally hoping for something that supported quantization for CPU inference. I explored llama.cpp and GGML, but ultimately landed on ctranslate2 for inference.

• CTranslate2: An efficient inference engine for Transformer models
  1 project | news.ycombinator.com | 21 May 2023
• A note from our sponsor - SaaSHub
  www.saashub.com | 19 Aug 2026

  SaaSHub helps you find the best software and product alternatives Learn more →