Build software better, together

This repository provides a basic implementation of self-attention. The code demonstrates how attention mechanisms work in predicting the next word in a sequence. It's a basic implementation that demonstrates the core concept of attention but lacks the complexity of more advanced models like Transformers.

python machine-learning ml python3 rnn attention-mechanism rnn-model attention-is-all-you-need self-attentive-rnn self-attention transformer-architecture self-attention-implementation

Updated Sep 23, 2024
Python

HEMANGANI / Music-Generation-Using-WGAN-GP-and-Self-Attention-Mechanism

Star

Developed a music generation deep learning model using WGAN-GP and self-attention, aimed at creating melodic compositions.

deep-learning pytorch generative-adversarial-network convolutional-neural-networks wgan-gp self-attention music-generation-deep-learning

Updated Sep 22, 2024
Jupyter Notebook

ChongQingNoSubway / DGR-MIL

Star

Code for paper: DGR-MIL: Exploring Diverse Global Representation in Multiple Instance Learning for Whole Slide Image Classification [ECCV 2024]

transformer weakly-supervised-learning whole-slide-imaging tumor-heterogeneity tumor-detection self-attention multiple-instance-learning tumor-classification histopathology histopathology-image-analysis

Updated Sep 21, 2024
Python

dcarpintero / generative-ai-101

Star

Annotated Notebooks to dive into Self-Attention, In-Context Learning, RAG, Knowledge-Graphs, Fine-Tuning, Model Optimization, and many more.

embeddings transformer knowledge-graph chunking bert fine-tuning self-attention model-quantization weights-and-biases huggingface-transformers in-context-learning large-language-models generative-ai langchain retrieval-augmented-generation llama3-1

Updated Sep 10, 2024
Jupyter Notebook

sparks-baird / CrabNet

Star

Predict materials properties using only the composition information!

python machine-learning pytorch materials-science materials-informatics attention-mechanism attention-is-all-you-need materials-screening self-attention materials-genome predict-materials-properties materials-discovery

Updated Sep 9, 2024
HTML

NVlabs / MambaVision

Star

Official PyTorch Implementation of MambaVision: A Hybrid Mamba-Transformer Vision Backbone

deep-learning transformers image-classification mamba visual-recognition self-attention hybrid-models huggingface-transformers vision-transformer foundation-models

Updated Sep 9, 2024
Python

datawhalechina / leedl-tutorial

Star

《李宏毅深度学习教程》（李宏毅老师推荐👍，苹果书🍎），PDF下载地址：https://github.com/datawhalechina/leedl-tutorial/releases

machine-learning tutorial reinforcement-learning deep-learning cnn transformer gan rnn pruning transfer-learning bert diffusion self-attention network-compression chatgpt leedl-tutorial

Updated Sep 9, 2024
Jupyter Notebook

Syeda-Farhat / awesome-Transformers-For-Segmentation

Star

Semantic segmentation is an important job in computer vision, and its applications have grown in popularity over the last decade.We grouped the publications that used various forms of segmentation in this repository. Particularly, every paper is built on a transformer.

computer-vision transformer segmentation semantic-segmentation encoder-decoder instance-segmentation multihead-attention self-attention

Updated Sep 6, 2024

aniketDash7 / self-attention-implementation

Star

Implementation of self attention in transformer architecture with the help of numpy

numpy transformers attention-mechanism self-attention transformer-architecture self-attention-implementation

Updated Sep 5, 2024
Jupyter Notebook

WenjieDu / SAITS

Sponsor

Star

The official PyTorch implementation of the paper "SAITS: Self-Attention-based Imputation for Time Series". A fast and state-of-the-art (SOTA) deep-learning neural network model for efficient time-series imputation (impute multivariate incomplete time series containing NaN missing data/values with machine learning). https://arxiv.org/abs/2202.08516