I have reimplemented Stable Diffusion 3.5 from scratch in pure PyTorch (github.com)

Article summary

The article discusses a reimplementation of the Stable Diffusion 3.5 model in pure PyTorch, called miniDiffusion, with minimal dependencies. The model is designed for educational, experimenting, and hacking purposes. The code is available on GitHub and includes implementations of VAE, CLIP, and T5 text encoders. The model uses weights from Hugging Face due to hardware constraints.

Main themes

• Stable Diffusion 3.5 reimplementation
• PyTorch
• Model architecture and weights
• Copyright and licensing
• Reference implementations and bugs
• Training and inference divergence
• Fine-tuning and experimentation
• Educational resources and learning

What commenters say

• The reimplementation of Stable Diffusion 3.5 is a valuable resource for learners and can be used for fine-tuning and inference.
• Reference implementations of models can be buggy and may not match the trained model code.
• The use of pre-trained weights from Hugging Face raises questions about the implementation and its compatibility with the original model.
• The distinction between model architecture, code, and weights is important in understanding copyright and licensing issues.
• The reimplementation may not be entirely compatible with the original model due to differences in training data and hardware constraints.
• Some commenters argue that the reimplementation is not a violation of the original license, while others disagree.
• The use of pre-trained weights and reference implementations can lead to training/inference divergence and bugs.
• Fixing bugs in reference implementations can improve results, but may not be straightforward.