Stable Diffusion with Core ML on Apple Silicon (machinelearning.apple.com)

Article summary

Apple has released optimizations for Core ML to run Stable Diffusion models on Apple Silicon devices, allowing for faster and more private image generation. The optimizations enable developers to deploy Stable Diffusion models on-device, reducing server-related costs and protecting user privacy. The release includes a Python package for converting Stable Diffusion models to Core ML and a Swift package for deployment. Benchmarks show significant performance improvements on Apple Silicon devices.

Main themes

• Stable Diffusion optimizations
• Apple Silicon performance
• On-device deployment
• Model versatility
• Video generation
• Cloud services
• GPU performance
• Privacy and cost considerations

What commenters say

• The new optimizations bring the performance of Apple Silicon devices in line with high-end GPUs like the GTX 1080.
• Some commenters believe that DALL-E's output quality is still superior to Stable Diffusion's, despite the performance improvements.
• Others argue that the community will catch up and improve Stable Diffusion's capabilities, making it a more viable option.
• The removal of certain training data from Stable Diffusion 2.0 has limited its versatility, but the community is working to adapt and improve the model.
• Some users are achieving faster performance with alternative builds and optimizations, such as using the MPS backend or thermal pad mods.
• The use of Stable Diffusion for video generation is possible but still faces challenges, such as maintaining temporal consistency.
• Cloud services like DreamStudio and Craiyon may be using high-end GPUs to achieve faster performance, but this comes at a cost.
• The performance differences between various Apple Silicon devices, such as the M1 and M2, are significant and impact the usability of Stable Diffusion.