LLM training code for Databricks foundation models

A high-throughput and memory-efficient inference and serving engine for LLMs

该仓库主要记录 大模型（LLMs） 算法工程师相关的面试题

📖A curated list of Awesome LLM Inference Paper with codes, TensorRT-LLM, vLLM, streaming-llm, AWQ, SmoothQuant, WINT8/4, Continuous Batching, FlashAttention, PagedAttention etc.

Finetuning Large Language Models on One Consumer GPU in Under 4 Bits

Large Language Model Text Generation Inference

Universal LLM Deployment Engine with ML Compilation

Easy-to-use and high-performance NLP and LLM framework based on MindSpore, compatible with models and datasets of 🤗Huggingface.

Hackable and optimized Transformers building blocks, supporting a composable construction.

LLM training in simple, raw C/CUDA

Hunyuan-DiT : A Powerful Multi-Resolution Diffusion Transformer with Fine-Grained Chinese Understanding

Mamba SSM architecture

TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and build TensorRT engines that contain state-of-the-art optimizations to perform inference efficie…

MII makes low-latency and high-throughput inference possible, powered by DeepSpeed.

LightLLM is a Python-based LLM (Large Language Model) inference and serving framework, notable for its lightweight design, easy scalability, and high-speed performance.

High-speed Large Language Model Serving on PCs with Consumer-grade GPUs

MindSpore online courses: Step into LLM

LLM101n: Let's build a Storyteller

OneDiff: An out-of-the-box acceleration library for diffusion models.