Efficient implementations of state-of-the-art linear attention models in Pytorch and Triton

Official implementation of "Samba: Simple Hybrid State Space Models for Efficient Unlimited Context Language Modeling"

Simplified Masked Diffusion Language Model

Pytorch implementation for ICML 2024 paper Proteus: Exploring Protein Structure Generation for Enhanced Designability and Efficiency.

Building blocks for foundation models.

A comprehensive benchmark on the performances of multiple protein backbone generative models.

Minimalistic, extremely fast, and hackable researcher's toolbench for GPT models in 307 lines of code. Reaches <3.8 validation loss on wikitext-103 on a single A100 in <100 seconds. Scales to large…

Open weights language model from Google DeepMind, based on Griffin.

LLM training in simple, raw C/CUDA

Fast modular code to create and train cutting edge LLMs

A MAD laboratory to improve AI architecture designs 🧪

Protein Structure Transformer (PST): Endowing pretrained protein language models with structural knowledge

Structured state space sequence models

Curated list of chatgpt prompts from the top-rated GPTs in the GPTs Store. Prompt Engineering, prompt attack & prompt protect. Advanced Prompt Engineering papers.

🐚 OpenDevin: Code Less, Make More

The Art of Debugging

Dataset and package for working with protein-protein interactions in 3D

Website for hosting the Open Foundation Models Cheat Sheet.

OpenDiT: An Easy, Fast and Memory-Efficient System for DiT Training and Inference

Compute various size metrics for a Git repository, flagging those that might cause problems

PyTorch code and models for V-JEPA self-supervised learning from video.

Repo for "Monarch Mixer: A Simple Sub-Quadratic GEMM-Based Architecture"

Code for the paper https://arxiv.org/abs/2402.04997

AlphaFold Meets Flow Matching for Generating Protein Ensembles

A mini-library for training consistency models.