Pinned

1. microsoft/DeepSpeed microsoft/DeepSpeed Public

   DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

   Python 33.2k 3.9k

2. microsoft/DeepSpeed-MII microsoft/DeepSpeed-MII Public

   MII makes low-latency and high-throughput inference possible, powered by DeepSpeed.

   Python 1.7k 160

3. alibaba/BladeDISC alibaba/BladeDISC Public

   BladeDISC is an end-to-end DynamIc Shape Compiler project for machine learning workloads.

   C++ 762 159

4. AlibabaResearch/flash-llm AlibabaResearch/flash-llm Public

   Flash-LLM: Enabling Cost-Effective and Highly-Efficient Large Generative Model Inference with Unstructured Sparsity

   Cuda 150 11

5. usyd-fsalab/fp6_llm usyd-fsalab/fp6_llm Public

   An efficient GPU support for LLM inference with x-bit quantization (e.g. FP6,FP5).

   Cuda 133 12

6. VersaPipe VersaPipe Public

   A framework for pipelined computing on GPU

   C++ 27 9