lucidrains

lucidrains

vit-pytorch

★ 25,506

Implementation of Vision Transformer, a simple way to achieve SOTA in vision classification with only a single transformer encoder, in Pytorch

lucidrains

denoising-diffusion-pytorch

★ 10,686

Implementation of Denoising Diffusion Probabilistic Model in Pytorch

lucidrains

PaLM-rlhf-pytorch

★ 7,862

Implementation of RLHF (Reinforcement Learning with Human Feedback) on top of the PaLM architecture. Basically ChatGPT but with PaLM

lucidrains

x-transformers

★ 5,945

A concise but complete full-attention transformer with a set of promising experimental features from various papers

lucidrains

vector-quantize-pytorch

★ 4,003

Vector (and Scalar) Quantization, in Pytorch

lucidrains

lion-pytorch

★ 2,203

🦁 Lion, new optimizer discovered by Google Brain using genetic algorithms that is purportedly better than Adam(w), in Pytorch

lucidrains

titans-pytorch

★ 1,981

Unofficial implementation of Titans, SOTA memory for transformers, in Pytorch

lucidrains

byol-pytorch

★ 1,901

Usable Implementation of "Bootstrap Your Own Latent" self-supervised learning, from Deepmind, in Pytorch

lucidrains

alphafold3-pytorch

★ 1,693

Implementation of Alphafold 3 from Google Deepmind in Pytorch

lucidrains

transfusion-pytorch

★ 1,402

Pytorch implementation of Transfusion, "Predict the Next Token and Diffuse Images with One Multi-Modal Model", from MetaAI

lucidrains

perceiver-pytorch

★ 1,217

Implementation of Perceiver, General Perception with Iterative Attention, in Pytorch

lucidrains

BS-RoFormer

★ 938

Implementation of Band Split Roformer, SOTA Attention network for music source separation out of ByteDance AI Labs

lucidrains

rotary-embedding-torch

★ 824

Implementation of Rotary Embeddings, from the Roformer paper, in Pytorch

lucidrains

ema-pytorch

★ 659

A simple way to keep track of an Exponential Moving Average (EMA) version of your Pytorch model

lucidrains

pi-zero-pytorch

★ 584

Implementation of π₀, the robotic foundation model architecture proposed by Physical Intelligence

lucidrains

enformer-pytorch

★ 573

Implementation of Enformer, Deepmind's attention network for predicting gene expression, in Pytorch

lucidrains

slot-attention

★ 498

Implementation of Slot Attention from GoogleAI

lucidrains

rectified-flow-pytorch

★ 484

Implementation of rectified flow and some of its followup research / improvements in Pytorch

lucidrains

autoregressive-diffusion-pytorch

★ 441

Implementation of Autoregressive Diffusion in Pytorch

lucidrains

q-transformer

★ 406

Implementation of Q-Transformer, Scalable Offline Reinforcement Learning via Autoregressive Q-Functions, out of Google Deepmind