PyTorch 1.10

This page summarizes the projects mentioned and recommended in the original post on news.ycombinator.com

Our great sponsors
  • WorkOS - The modern identity platform for B2B SaaS
  • InfluxDB - Power Real-Time Data Analytics at Scale
  • SaaSHub - Software Alternatives and Reviews
  • vision

    Datasets, Transforms and Models specific to Computer Vision

    haha, yes, but that requires you to modify existing code to do so (which isn't always possible!).

    There might also be other things you want to do (like add profiling after each op) that would be tedious to do manually, but can easily automated with FX (https://pytorch.org/tutorials/intermediate/fx_profiling_tuto...).

    Another example is the recent support from torchvision for extracting intermediate feature activations (https://github.com/pytorch/vision/releases/tag/v0.11.0). Like, sure, it was probably possible to refactor all of their code to enable users to specify extracting an intermediate feature, but it's much cleaner to do with FX.

  • onnxruntime

    ONNX Runtime: cross-platform, high performance ML inferencing and training accelerator

    Any recommendations about how to use a PyTorch trained model for inference? Is it best to load it up with PyTorch directly, or convert to ONNX and use ONNX-runtime [1] instead? This seems to be the required way at least if you want to TensorRT the model. I appreciate this is a very general question.

    [1] https://github.com/microsoft/onnxruntime

  • WorkOS

    The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

  • nn

    🧑‍🏫 60 Implementations/tutorials of deep learning papers with side-by-side notes 📝; including transformers (original, xl, switch, feedback, vit, ...), optimizers (adam, adabelief, sophia, ...), gans(cyclegan, stylegan2, ...), 🎮 reinforcement learning (ppo, dqn), capsnet, distillation, ... 🧠

    We did a bunch of popular research paper implementations in PyTorch with notes (annotations); might be helpful.

    https://nn.labml.ai

  • onnx-simplifier

    Simplify your onnx model

    As far as I know, the ONNX format won't give you a performance boost on its own. However, there are ONNX optimizers for the ONNX runtime which will speed up your inference.

    But if you are using Nvidia Hardware, then TensorRT should give you the best performance possible, especially if you change the precision level. Don't forget to simplify your ONNX model before you converting it to TensorRT though: https://github.com/daquexian/onnx-simplifier

  • torch2trt

    An easy to use PyTorch to TensorRT converter

    Main thing you want for server inference is auto batching. It's a feature that's included in onnxruntime, torchserve, nvidia triton inference server and ray serve.

    If you have a lot of preprocessing and post logic in your model it can be hard to export it for onnxruntime or triton so I usually recommend starting with Ray Serve (https://docs.ray.io/en/latest/serve/index.html) and using an actor that runs inference with a quantized model or optimized with tensorrt (https://github.com/NVIDIA-AI-IOT/torch2trt)

  • TensorRT

    PyTorch/TorchScript/FX compiler for NVIDIA GPUs using TensorRT (by pytorch)

    You can quantize your model to FP16 or Int8 using PTQ as well and it should give you an additional speed up inference wise.

    Here is a tutorial[2] to leverage TRTorch.

    [1] https://github.com/NVIDIA/TRTorch/tree/master/core

  • functorch

    functorch is JAX-like composable function transforms for PyTorch.

    https://github.com/pytorch/functorch) but not the second.

    Disclaimer: I work on PyTorch, and Functorch more specifically, although my opinions here aren't on behalf of PyTorch.

  • InfluxDB

    Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts