Generating QR Code with Stable Diffusion

A recent Reddit post showcased a series of artistic QR codes created with Stable Diffusion. Those QR codes were generated with a custom-trainedControlNet model. Just like another day in the Stable Diffusion community, people have quickly figured out how to generate QR codes with Stable Diffusion WITHOUT a custom model.

QR code, short for Quick Response code, is a common way to encode text or URL in a 2D image. You can typically use your phone’s camera app to read the code.

In this post, you will learn how to generate QR codes like these.

We will study a few ways to generate QR codes with Stable Diffusion.

  1. Use a QR Code Control Model with text-to-image.
  2. Use the tile Control Model with image-to-image.
  3. Use a QR Code Control Model with image-to-image.

Method 1 is easier to control, and the color comes out brighter. But not all prompts work.

Method 2 allows a wider range of prompts but could have color issues.

Method 3 produces image quality between methods 1 and 2.

Software

We will use AUTOMATIC1111 Stable Diffusion GUI to create QR codes. You can use this GUI on Google ColabWindows, or Mac.

You will need the ControlNet extension installed. Follow this tutorial to install.

If you are using our Colab Notebook, simply select ControlNet at startup.

Generating QR code

You will first need a QR Code. To increase your chance of success, use a QR code that meets the following criteria.

  • Use a high fault tolerance setting (30%).
  • Have a white margin around the QR Code (the quiet zone).
  • Use the most basic square fill with a black-and-white pattern.
  • Avoid using generators that introduce a thin white line between black elements.

We will use this QR Code generator in this tutorial.

Step 1: Select the text type and enter the text for the QR code.

Step 2: Set fault tolerance to 30%.

Step 3: Press Generate.

Step 4: Download the QR Code as a PNG file.

Method 1: Generate a QR code with a QR control model in text-to-image

This method requires you to download a ControlNet model trained for decorating QR codes. We have two choices.

Step 1: Install the QR Code Control Model.

We will use QR code Pattern v2.

Download the ControlNet Model and put it in the following folder.

stable-diffusion-webuiextensionssd-webui-controlnetmodels

Step 2: Enter the text-to-image setting

Stable Diffusion checkpoint: Anything v3

The prompt is quite important to your success. Some prompts blend naturally with your QR Code.

Prompt:

Japanese painting, mountains, 1girl

Negative Prompt:

ugly, disfigured, low quality, blurry, nsfw

  • Sampling Method: DPM++ 2M Karras
  • SamplingStep: 20
  • Hires Fix: On – Upscaler: Latent. Upscaleby 2. Denoisingstrength: 0.7
  • Width: 512
  • Height: 512
  • CFGScale: 7

Step 3: Enter ControlNet Setting

On the text2img page, expand the ControlNet section. You should see an empty image canvas under ControlNet Unit 0 > Single Image.

Upload the QR code to the Image Canvas.

Enable: Yes

Pixel Perfect: Yes

ControlNet Model: controlnetQRPatternQR_v2Sd15

Control Weight: 1.1 (This needs to be adjusted for each prompt)

Control Mode: Balanced

Resize Mode: Crop and Resize

Step 4: Press Generate

You should get 1024×1024 beautiful QR codes like these:

Method 2: Generate a QR code with the tile resample model in image-to-image

This method starts with generating an image similar to the QR Code using img2img. But this is not enough to produce a valid QR Code. ControlNet is turned on during the sampling steps to imprint the QR code onto the image. Near the end of the sampling steps, ControlNet is turned off to improve the consistency of the image.

In AUTOMATIC1111 WebUI, navigate to the Img2img page.

Step 1: Select a checkpoint model.

We will use GhostMix.

Step 2: Enter a prompt and a negative prompt.

The prompt is quite important to your success. Some prompts blend naturally with your QR Code.

We will use the following prompt.

a cubism painting of a town with a lot of houses in the snow with a sky background, Andreas Rocha, matte painting concept art, a detailed matte painting

And the following negative prompt.

ugly, disfigured, low quality, blurry, nsfw

Step 3: Upload the QR code to the img2img canvas.

Step 4: Enter the following image-to-image settings.

  • Resize mode: Just resize
  • Sampling method: DPM++2M Karras
  • Sampling step: 50
  • Width: 768
  • Height: 768
  • CFG Scale: 7
  • Denoising strength: 0.75

Step 5: Upload the QR code to ControlNet‘s image canvas.

Step 6: Enter the following ControlNet settings.

  • Enable: Yes
  • Control Type: Tile
  • Preprocessor: tile_resample
  • Model: control_xxx_tile
  • Control Weight: 0.87
  • Starting Control Step: 0.23
  • Ending Control Step: 0.9

Step 7: Press Generate.

Step 6: Check the QR code with your phone. Make sure to check with different sizes on the screen. Some tend to have issues when they are large.

You won’t get a functional QR Code with every single image. The success rate is about one in four. Generate more images and check for the keepers.

Tips

QR codes with shorter text have a higher success rate because the patterns are simpler.

Not all QR codes work the same. Some could be marginally working and can only be read at a certain distance.

Some prompts blend more naturally with QR codes. For example, the prompt for generating houses with snow on rooftops you saw previously blends well with QR codes simply because they share similar visual elements.

The working parameters can be different for different models and prompts. You must adjust the following parameter slightly to blend the QR Code and the prompt well.

  • Denoising strength: Decrease to have the initial composition follows the QR code more. But you will only see the QR code if you reduce it too much. It is typically set higher than 0.7.
  • Control Weight: Decrease to show the prompt more.
  • Starting Control Step: Increase to show the prompt more.
  • Ending Control Step: Decrease to stop the ControlNet earlier so that the QR code and the image can blend more naturally.

Method 3: Generate a QR code with the QR control model in image-to-image

This method is very similar to Method 2, except we chose the QR Pattern v2 model instead of the tile Control Model. Here are the parameters.

Prompt:

A photo-realistic rendering of a busy market, ((street vendors, fruits, vegetable, shops)), (Photorealistic:1.3), (Highly detailed:1.2), (Natural light:1.2), art inspired by Architectural Digest, Vogue Living, and Elle Decor

Negative prompt:

ugly, disfigured, low quality, blurry, nsfw

  • Enable: Yes
  • Control Type: All
  • Preprocessor: None
  • Model: controlnetQRPatternQR_v2Sd15
  • Control Weight: 1
  • Starting Control Step: 0.12 (Adjust for different prompts)
  • Ending Control Step: 1

Here’s what we got:

Other prompts

In Snow

masterpiece, best quality, winter, snow field, smile, blush, 1girl, bangs, blue eyes, blunt bangs, bonnet, brown footwear, brown hair, red dress, frills, fruit, full body, hat, long hair, long sleeves, looking at viewer, black pantyhose, red flower, red rose, rose, shoes, sitting, solo, sky, sunset, mountain, forest, lake,

EasyNegative, (worst quality:2), (low quality:2), (normal quality:2) verybadimagenegative_v1.3

Method 3

Model: Ghost Mix

Start Control Step: 0.12

Ending Control Step: 1

Mechanical girl

Method 2

1mechanical girl,ultra realistic details, portrait, global illumination, shadows, octane render, 8k, ultra sharp,intricate, ornaments detailed, cold colors, metal, egypician detail, highly intricate details, realistic light, trending on cgsociety, glowing eyes, facing camera, neon details, machanical limbs,blood vessels connected to tubes,mechanical vertebra attaching to back,mechanical cervial attaching to neck,sitting,wires and cables connecting to head

ugly, disfigured, low quality, blurry

Denoising strength: 0.75

Control weight: 1

Starting Control Step: 0.23

Ending Control Step: 0.7

Japanese girl

Japanese painting, mountains, 1girl

ugly, disfigured, low quality, blurry, nsfw

Method 1

Control Weight: 1.1

Method 2

Denoising strength: 0.75

Control weight: 1

Starting Control Step: 0.23

Ending Control Step: 0.9

Robot

light, futobot, cyborg, ((masterpiece),(best quality),(ultra-detailed), (full body:1.2), 1male, solo, hood up, upper body, mask, 1boy, male focus,white gloves, cloak, long sleeves, spaceship, lightning, hires

ugly, disfigured, low quality, blurry

Method 2

Denoising strength: 0.75

Control weight: 1

Starting Control Step: 0.24

Ending Control Step: 0.9

Outdoor market

A photo-realistic rendering of a busy market, ((street vendors, fruits, vegetable, shops)), (Photorealistic:1.3), (Highly detailed:1.2), (Natural light:1.2), art inspired by Architectural Digest, Vogue Living, and Elle Decor

ugly, disfigured, low quality, blurry, nsfw

Denoising strength: 0.75

Control weight: 0.87

Starting Control Step: 0.23

Ending Control Step: 0.9

Some interesting reads

aizmin: