Removing Background in Stable Diffusion

Many free online apps let you remove background of an image for free. But you may not want to use them because of privacy concerns.

In this post, you will learn how to use a Stable Diffusion GUI to remove the background of any image locally on your computer. You will have total control of how your images are stored. As a bonus, you can use some advanced options to optimize the background removal.

Software

1. Start AUTOMATIC1111 Web-UI normally.

2. Navigate to the Extension Page.

3. Click the Install from URL tab.

4. Enter the following URL in the URL for extension’s git repository field.

https://github.com/AUTOMATIC1111/stable-diffusion-webui-rembg

5. Click the Install button.

6. Wait for the confirmation message that the installation is complete.

7. Restart AUTOMATIC1111.

Remove background in Stable Diffusion

After installing the background removal extension successfully, you will see a new Remove Background menu on the Extras page in AUTOMATIC1111. (Now the Extras page finally makes sense…)

You can use it to remove the background of any image, whether it is real or generated by AI.

We will go through the steps to remove the background of an image generated by Stable Diffusion. It is easy to extend this workflow to a real image.

Step 1: Generate an AI image

First, generate an image on the txt2img page in AUTOMATIC1111. I will use the following settings, but you can use this workflow for any images.

• Stable Diffusion Checkpoint: JuggernautXL
• Prompt:

samurai girl, 18 year old, detailed eyes, white and red kimono, strange but extremely beautiful, mysterious, fearsome, full body

• Negative prompt:

disfigured, ugly, bad, immature, cartoon, anime, 3d, painting, b&w, 2d, 3d, illustration, sketch, nfsw, nude

• Sampling method: DPM++ 2M Karras
• Sampling steps: 20
• Size: 832 x 1216
• CFG Scale: 7

Select the image you want to remove the background. Click the Send to Extras button under the image to send it to the Extras page.

You should now be on the Extras page. Your image should be in the Source Canvas.

Alternatively, you can drag and drop any image to the Source Canvas.

Step 2: Remove background

Select a background removal model in the Remove Background dropdown menu. u2net works well for most images.

You can optionally upscale the image at the same time. Leaving the Upscalers to None disables them.

Press Generate to remove the background. Now, you get an image with the background removed.

Advanced options

Return mask

Checking the Return mask option generates a black-and-white mask instead of the image. It is useful to use as an inpainting mask for further processing, such as adding a new background. But I will save it for the next post!

Alpha matting

You may have noticed parts of her headwear are incorrectly removed. You can get them back by adjusting the alpha matting settings.

An alpha matte is a pixel map that separates the foreground and the background. The task of background removal is finding an accurate alpha matte.

These options let you adjust how the alpha matte is created.

Foreground threshold

Reduce the foreground threshold to expand the foreground.

Background threshold

Reduce the background threshold to expand the foreground.

Erode size

Reduce the Erode size to increase fine details on the boundary.

Background models

In my experience, u2net works for most images. It is a good starting point for further adjusting the alpha matte.

If you find u2net fails drastically, try other models on the list.