A native PyTorch Library for large model training

Example of binding a TF32 CUTLASS GEMM kernel to PyTorch

SpeeD: A Closer Look at Time Steps is Worthy of Triple Speed-Up for Diffusion Model Training

Making large AI models cheaper, faster and more accessible

A PyTorch Extension: Tools for easy mixed precision and distributed training in Pytorch

An efficient GPU support for LLM inference with 6-bit quantization (FP6).

ActNN: Reducing Training Memory Footprint via 2-Bit Activation Compressed Training

A Easy-to-understand TensorOp Matmul Tutorial

<forked> Tensors and Dynamic neural networks in Python with strong GPU acceleration

GPU implementation of a fast generalized ANS (asymmetric numeral system) entropy encoder and decoder, with extensions for lossless compression of numerical and other data types in HPC/ML applications.

A Native-PyTorch Library for LLM Fine-tuning

This repository contains the experimental PyTorch native float8 training UX

Repository for nvCOMP docs and examples. nvCOMP is a library for fast lossless compression/decompression on the GPU that can be downloaded from https://developer.nvidia.com/nvcomp.

A byte-level decoder architecture that matches the performance of tokenized Transformers.

CUDA Templates for Linear Algebra Subroutines

Reference implementation of Megalodon 7B model

A high-throughput and memory-efficient inference and serving engine for LLMs

Step-by-step optimization of CUDA SGEMM

FP16xINT4 LLM inference kernel that can achieve near-ideal ~4x speedups up to medium batchsizes of 16-32 tokens.

Pipeline Parallelism for PyTorch