Raspberry Pi 5 Review: Running Stable Diffusion Model for AI Image Generation

auroraAA Jan 24.2024

0 773 Medium

Stable Diffusion (SD) is a 2022 deep learning model based on diffusion techniques, contributing to the ongoing AI spring. It excels in generating detailed images from text descriptions and versatile tasks guided by textual prompts. As a latent diffusion model, SD belongs to the family of deep generative artificial neural networks. Its open-sourced code and model weights enable efficient running on consumer hardware with a modest GPU (minimum 4 GB VRAM), distinguishing it from previous proprietary models like DALL-E and Midjourney, accessible only through cloud services.

This article describes an experimental procedure for running the SD model on a Raspberry Pi 5. Although the performance of the Raspberry Pi limits the efficiency of SD, this attempt is valuable for learning and experimentation. The experimental steps covered cloning the SD model, compiling the XNNPACK tool, compiling the SD example, and running the model on the Raspberry Pi 5. Specific command line operations and parameter settings are also covered in detail, as well as resource usage and generated results during the experiment.

Experiment conditions

- System : Raspberry Pi 5 with latest Raspberry Pi OS x64
- Hardware: Use the 8GB version of the Raspberry Pi 5, with an official active heatsink and a 5V4A power adapter
- Network: The model to be downloaded is large, approximately 16GB, and requires a very stable internet connection

1. Experiment on Stable Diffusion XL 1.0

1.1 Cloning of Stable Diffusion model: Cloning of Stable Diffusion XL 1.0 model from here.

Using commands:
//Raspberry Pi default git does not support lfs, you need to install additional dependencies

//Add Warehouse
curl -s script.deb.sh | sudo bash

//install
sudo apt-get install git-lfs

//Activate
git lfs install

//Extract this model
t clone --depth=1 link

This step is very time-consuming, make sure the download speed is normal, you can wait a little bit and then come back to check, this step is recommended to open the terminal directly in the Raspberry Pi (or VNC), to prevent the computer's ssh break.

1.2 Compile XNNPACK tool

# If you don't have Cmake, you need to install it first.
sudo apt install cmake

# Install Warehouse
git clone [https://github.com/google/XNNPACK.gitcd XNNPACK]

# Find the item compatible with the
commitgit rev-list -n 1 --before="2023-06-27 00:00" master

# Here you can get a string (Commit id) and paste it into the next command
git checkout <COMMIT_ID obtained in the previous step>

# compile
mkdir buildcd buildcmake -DXNNPACK_BUILD_TESTS=OFF -DXNNPACK_BUILD_BENCHMARKS=OFF .. cmake --build . --config Release make

# Check that there should be a libXNNPACK.a file in the current directory.
ls

1.3 Compile Stable Diffusion exmaple

# Install Warehouse
git clone gitcd OnnxStreamcd srcmkdir buildcd build

# Compile，Here, since we are using a Raspberry Pi 5, here we use the DMAX_SPEED=ON，-DXNNPACK_DIR, then attach it to the XNNPACK location in the previous step, and modify it here.
cmake -DMAX_SPEED=ON -DXNNPACK_DIR=/home/pi/xxx/XNNPACK ..
cmake --build . --config Release

# To check, there should be a file named sd in the current directory
ls

1.4 Start diffusion, try SD XL 1.0 here first!

# Use this part of the argument here
# --xl run stable-diffusion XL 1.0
# ---models-path Set the folder containing the stable-diffusion models.
# --output Set the path to the output PNG file.
# --prompt Set the forward prompt
# --steps Set the number of diffusion steps
# --ram Use RAM WeightsProvider to allow large memory usage, tested up to 5.3G memory footprint.
# --rpi Configure the model to run on a normal Raspberry Pi, e.g. Raspberry Pi 3, 4, 5.

# Run
. /sd --xl ---models-path /home/pi/xxxx/stable-diffusion-xl-base-1.0-onnxstream --output result.png --prompt "A photo of an astronaut riding a horse" --steps 3 --ram --rpi