Stable diffusion 2.1 was released on Dec 7, 2022.

Those who have used 2.0 have been scratching their head on how to make the most of it. While we see some excellent images here or there, most of us went back to v1.5 for their business.

See the step-by-step guide for installing AUTOMATIC1111 on Windows.

The difficulty was in part caused by (1) using a new language model that is trained from scratch, and (2) the training dataset was heavily censored with an NSFW filter.

The second part would have been fine, but the filter was quite inclusive and removed a substantial amount of good-quality data. 2.1 promised to bring them back.

This tutorial will cover installing and using 2.1 models in AUTOMATIC1111 GUI so you can judge by using it.

2.1 models variants

There are two text-to-image models available:

• 2.1 base model: Default image size is 512×512 pixels
• 2.1 model: Default image size is 768×768 pixels

The 768 model is capable of generating larger images. You can set the image size to 768×768 without worrying about the infamous two heads issue.

This is especially useful for generating larger scenes with small characters. The faces can be generated a bit clearer than the 512 model, increasing the chance of success of the downstream upscaling and face restoration.

The downside of the 768 model is it takes longer to generate images. The larger images may limit the batch size, depending on how much VRAM your GPU has.

Install base software

We will go through how to use Stable Diffusion 2.0 in AUTOMATIC1111 GUI.

This GUI can be installed easily in Windows systems, or follow the installation instructions on your respective environment. Ideally, you should have a dedicated GPU card with at least 6GB VRAM.

If you already have this GUI, make sure it is up-to-date by running the following command in the terminal under its installation location (stable-diffusion-webui folder).

git pull

Download Stable Diffusion 2.1 model

2.1 base model (512-base)

1. Download the model file (v2-1_512-ema-pruned.ckpt)

https://huggingface.co/stabilityai/stable-diffusion-2-1-base/resolve/main/v2-1_512-ema-pruned.ckpt

2. Download the config file, and rename it to v2-1_512-ema-pruned.yaml

https://raw.githubusercontent.com/Stability-AI/stablediffusion/main/configs/stable-diffusion/v2-inference.yaml

Put both of them in the model directory:

stable-diffusion-webui/models/Stable-diffusion

2.1 model (768)

1. Download the model file (v2-1_768-ema-pruned.ckpt)

https://huggingface.co/stabilityai/stable-diffusion-2-1/resolve/main/v2-1_768-ema-pruned.ckpt

2. Download the config file, rename it to v2-1_768-ema-pruned.yaml

https://raw.githubusercontent.com/Stability-AI/stablediffusion/main/configs/stable-diffusion/v2-inference.yaml

Put both of them in the model directory:

stable-diffusion-webui/models/Stable-diffusion

How to use 2.1 model

To use the 768 version of the Stable Diffusion 2.1 model, select v2-1_768-ema-pruned.ckpt in the Stable Diffusion checkpoint dropdown menu on the top left.

The model is designed to generate 768×768 images. So, set the image width and/or height to 768 for the best result.

To use the base model, select v2-1_512-ema-pruned.ckpt instead.

Troubleshooting

This is something you can try if your installation doesn’t work.

• See if your AUTOMATIC1111 GUI is outdated. In the terminal, use the command git pull under the stable-diffusion-webui directory and restart the GUI.
• Check if the YAML file is downloaded correctly. Its content should be a simple text file, not with HTML tags.
• Check if the YAML file is correctly renamed as described in the previous section.
• If 2.0 or 2.1 is generating black images, enable full precision with startup arguments --no-half or --xformers optimization.

Tips for using 2.1

2.1 is an improvement over 2.0. The images look better and require less effort in engineering the prompt.

So, I am deleting my 2.0 models.

Here are some tips for using the 2.1 model.

Tip 1: Write more

Similar to 2.0, the prompt needs to be very specific and detailed to get the image you want. Unlike v1 models, simple prompts usually won’t go well with 2.1.

Tip 2: Use negative prompt

Many have already found that a negative prompt is important for v2 models. Keep a boilerplate negative prompt for portraits where many things can go wrong. In fact, Stability uses the following negative prompt in the demo images.

cropped, lowres, poorly drawn face, out of frame, poorly drawn hands, blurry, bad art, blurred, text, watermark, disfigured, deformed, closed eyes

I like the following negative prompt for the v2.1 model.

ugly, tiling, poorly drawn hands, poorly drawn feet, poorly drawn face, out of frame, extra limbs, disfigured, deformed, body out of frame, bad anatomy, watermark, signature, cut off, low contrast, underexposed, overexposed, bad art, beginner, amateur, distorted face

See how it is constructed and the importance of a negative prompt for the v2 models.

Tip 3: Use the correct image size

Finally, set the correct image size. Set at least one side to be 512 px for the 512-base model and 768 px for the 768 model.

Have fun with 2.1!