a gaggle of deep neural architectures for text ranking and question answering, designed for Pyserini

Must-read papers on prompt-based tuning for pre-trained language models.

Robust Speech Recognition via Large-Scale Weak Supervision

Pytorch implementation and extension of "DocUnet: Document Image Unwarping via A Stacked U-Net"

Unsupervised text tokenizer focused on computational efficiency

🧑‍🏫 60+ Implementations/tutorials of deep learning papers with side-by-side notes 📝; including transformers (original, xl, switch, feedback, vit, ...), optimizers (adam, adabelief, sophia, ...), ga…

Code for the paper "Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer"

Unsupervised text tokenizer for Neural Network-based text generation.

Minimalist BERT implementation assignment for CS11-711

An unofficial implementation of Poly-encoder (Poly-encoders: Transformer Architectures and Pre-training Strategies for Fast and Accurate Multi-sentence Scoring)

Long Range Arena for Benchmarking Efficient Transformers

Implementation of H-Transformer-1D, Hierarchical Attention for Sequence Learning

Transformer based on a variant of attention that is linear complexity in respect to sequence length

This repository contains the code for "Exploiting Cloze Questions for Few-Shot Text Classification and Natural Language Inference"

Some notebooks for NLP

The implementation of DeBERTa

MPNet: Masked and Permuted Pre-training for Language Understanding https://arxiv.org/pdf/2004.09297.pdf

[EMNLP 2021] SimCSE: Simple Contrastive Learning of Sentence Embeddings https://arxiv.org/abs/2104.08821