Hysen Labsstable-diffusion
A latent text-to-image diffusion model
Stable Diffusion, a latent text-to-image model
Stable Diffusion is a latent text-to-image diffusion model trained on 512x512 images from a subset of the LAION-5B database. The README covers the architecture, the weights, and the sampling scripts.
How it came together
The README credits a collaboration with Stability AI and Runway, building on earlier latent diffusion work, with a paper at CVPR '22 on high-resolution image synthesis. A compute donation from Stability AI and support from LAION made the training possible. The result is a latent diffusion model trained on 512x512 images from a subset of the LAION-5B database.
The architecture
The model conditions on text prompts through a frozen CLIP ViT-L/14 text encoder, similar to Google's Imagen. The UNet weighs in at 860M parameters and the text encoder at 123M, which the README calls relatively lightweight. It runs on a GPU with at least 10GB of VRAM. The v1 configuration uses a downsampling-factor 8 autoencoder, pretrained at 256x256 and then fine-tuned at 512x512.
Weights and licensing
Four v1 checkpoints are listed: sd-v1-1, sd-v1-2, sd-v1-3, and sd-v1-4, each with its own training steps and dataset notes. The weights live with the CompVis organization on Hugging Face under the CreativeML OpenRAIL M license, which carries use-based restrictions intended to prevent misuse. Commercial use is permitted, but the README advises against shipping services on these weights without additional safety mechanisms, since the model mirrors biases in its training data.
Sampling scripts
The reference sampling script defaults to a guidance scale of 7.5, Katherine Crowson's PLMS sampler, and 50 steps to render 512x512 images. A safety checker reduces the probability of explicit outputs, and invisible watermarking marks images as machine-generated. The README also points to a diffusers integration. The inference config is designed for EMA-only checkpoints, with full checkpoints available for comparing EMA and non-EMA weights.
Image modification and credits
A diffusion-denoising mechanism, first proposed in SDEdit, lets the model do text-guided image-to-image translation and upscaling. A strength value between 0.0 and 1.0 controls how much noise gets added to the input. The codebase builds on OpenAI's ADM codebase and lucidrains' denoising-diffusion-pytorch, with thanks noted in the README.
Community notes