Megatron-LM VS TensorRT

Compare Megatron-LM vs TensorRT and see what are their differences.


Ongoing research training transformer models at scale (by NVIDIA)


NVIDIA® TensorRT™, an SDK for high-performance deep learning inference, includes a deep learning inference optimizer and runtime that delivers low latency and high throughput for inference applications. (by NVIDIA)
Our great sponsors
  • Onboard AI - Learn any GitHub repo in 59 seconds
  • InfluxDB - Collect and Analyze Billions of Data Points in Real Time
  • SaaSHub - Software Alternatives and Reviews
Megatron-LM TensorRT
15 20
6,877 8,198
6.5% 3.9%
0.0 7.4
10 days ago 20 days ago
Python C++
BSD 3-clause "New" or "Revised" License Apache License 2.0
The number of mentions indicates the total number of mentions that we've tracked plus the number of user suggested alternatives.
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.


Posts with mentions or reviews of Megatron-LM. We have used some of these posts to build our list of alternatives and similar projects. The last one was on 2023-10-10.


Posts with mentions or reviews of TensorRT. We have used some of these posts to build our list of alternatives and similar projects. The last one was on 2023-09-26.

What are some alternatives?

When comparing Megatron-LM and TensorRT you can also consider the following projects:

DeepSpeed - DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

onnx-tensorrt - ONNX-TensorRT: TensorRT backend for ONNX

FasterTransformer - Transformer related optimization, including BERT, GPT

flash-attention - Fast and memory-efficient exact attention

stable-diffusion-webui - Stable Diffusion web UI

openvino - OpenVINO™ is an open-source toolkit for optimizing and deploying AI inference

ColossalAI - Making large AI models cheaper, faster and more accessible

tensorrtx - Implementation of popular deep learning networks with TensorRT network definition API

tvm - Open deep learning compiler stack for cpu, gpu and specialized accelerators

vllm - A high-throughput and memory-efficient inference and serving engine for LLMs

transformer-deploy - Efficient, scalable and enterprise-grade CPU/GPU inference server for 🤗 Hugging Face transformer models 🚀