Tips for Creating Engaging Prompts for Stable Diffusion

Prompt building is a basic skill any Stable Diffusion user should master. By understanding how to construct clear and concise prompts, you can unlock the whole range of style Stable Diffusion offers. To excel in prompt building, you should start with a specific subject in mind and add keywords to steer towards a particular effect.

The skill you learn in this article can be applied to other AI art generators such as MidJourney.

This is part 2 of the beginner’s guide series.
Read part 1: Absolute beginner’s guide.
Read part 3: Inpainting.
Read part 4: Models.

Resources for beginners

If you want to practice prompt building but do not have your Stable Diffusion set up yet, you can use a free Stable Diffusion generator online.

Read the Quick Start Guide if you want to set up your own.

Use the prompt builder for a systematic approach to craft prompts.

This post is intended to be your first course in prompting. See this prompt guide for advanced techniques.

Anatomy of a good prompt

This is a proven technique to generate high-quality, specific images. Your prompt should cover most, if not all, of these areas.

  1. Subject (required)
  2. Medium
  3. Style
  4. Artist
  5. Website
  6. Resolution
  7. Additional details
  8. Color
  9. Lighting

Subject

First, you will need a subject description with as much detail as possible. Below is an example.

Prompt (subject only):

A young woman with light blue dress sitting next to a wooden window reading a book.

A common mistake of a beginner is not describing the image in enough detail. What clothing does she wear? What’s her hairstyle? These seemingly minor elements can contribute significantly to the overall image being conveyed. Without specifying these details, you leave them open to the AI generator, and you may be disappointed about what you get.

It is a good practice to include a generic negative prompt.

ugly, deform, disfigured

Let’s use the Realistic Vision v2 model.

We got the following image, which matches the prompt pretty well.

Not bad. This model defaults to generate photo-realistic images, so this is the style you get.

Medium

We can be more specific. Let’s add a medium, the material that the artwork is created on. Some examples are digital painting, photography, and oil painting. Let’s use

Medium

Digital painting

The new prompt is

Digital painting of a young woman with light blue dress sitting next to a wooden window reading a book

The resulting image is

You can see the image changes from a photograph to a digital art. It’s a step forward but we can do better.

Adding the rest

You get the idea. Let’s add the rest of them

Artist – specifying the artist who created the art to steer the style.

by Stanley Artgerm Lau

Website – The name of the website can be used for a particular genre.

artstation

Resolution – These are keywords that control the sharpness of the image.

8k

Additional details – These are keywords that are more like sweeteners,e.g. adding some interesting details.

extremely detailed, ornate

LightingControling light is important for a good image.

cinematic lighting, rim lighting

color – The color scheme of the image

vivid

Putting them all together, the prompt is

Digital painting of a young woman with light blue dress sitting next to a wooden window reading a book, by Stanley Artgerm Lau, artstation, 8k, extremely detailed, ornate, cinematic lighting, rim lighting, vivid

Now we get this image.

We can engineer the image to get the style we want by adding specific keywords to the prompt.

Tips for good prompts

  • Be detailed and specific when describing the subject.
  • Use multiple brackets () to increase its strength and [] to reduce.
  • Use an appropriate medium type consistent with the artist. E.g. photograph should not be used with van Gogh.
  • The artist’s name is a very strong style modifier. Use wisely.
  • Experiment with blending styles.
  • Head to the Workflow section to study the high-quality prompts. Use the prompt as a starting point if you like a particular image.

Some good keywords for you

Below are some of my favorite keywords and their effects. (Used with Stable Diffusion v1.4 and v1.5). Enjoy!

You can find the full list in the prompt builder.

Can’t find the right words? Get the FULL LIST with 250+ well-tested keywords.

Medium

Medium defines a category of artwork.

keyword Note
Portrait Very realistic drawings. Good to use with people.
Digital painting Digital art style.
Concept art Illustration style, 2D.
Ultra realistic illustration Drawings that are very realistic. Good to use with people.
Underwater portrait Use with people. Underwater. Hair floating.
Underwater steampunk Very realistic drawings. Good to use with people.

Style

These keywords further refine the art style.

keyword Note
hyperrealistic Increases details and resolution
pop-art Pop-art style
Modernist vibrant color, high contrast
art nouveau Add ornaments and details, building style

Artist

Mentioning the artist in the prompt is a strong effect. Study their work and choose wisely.

keyword Note
John Collier 19th century portrait painter. Add elegancy
Stanley Artgerm Lau Good to use with woman portrait, generate 19th delicate clothing, some impressionism
Frida Kahlo Quite strong effect following Kahlo’s portrait style. Sometimes result in picture frame
John Singer Sargent Good to use with woman portraits, generate 19th delicate clothing, some impressionism
Alphonse Mucha 2D portrait painting in style of Alphonse Mucha

Website

Mentioning an art or photo site is a strong effect, probably because each site has its niche genre.

keyword Note
pixiv Japanese anime style
pixabay Commercial stock photo style
artstation Modern illustration, fantasy

Resolution

keyword Note
unreal engine Very realistic and detailed 3D
sharp focus Increase resolution
8k Increase resolution, though can lead to it looking more fake. Makes the image more camera like and realistic
vray 3D rendering best for objects, landscape and building.

Lighting

keyword Note
rim lighting light on edge of an object
cinematic lighting A generic term to improve contrast by using light
crepuscular rays sunlight breaking through the cloud

Additional details

Add specific details to your image.

keyword Note
dramatic shot from a low angle
silk Add silk to clothing
expansive More open background, smaller subject
low angle shot shot from low angle
god rays sunlight breaking through the cloud
psychedelic vivid color with distortion

Color

Add an additional color scheme to the image.

keyword Note
iridescent gold Shinny gold
silver Silver color
vintage vintage effect

Summary

We have gone through the basic structure of a good prompt. This should be used as a guide rather than a rule. The Stable Diffusion model is very flexible. Let it surprise you with some creative combination of keywords!

Check out the Stable Diffusion Course for a step-by-step guided course.

Or continue to part 3 below.

This is part 2 of the beginner’s guide series.
Read part 1: Absolute beginner’s guide.
Read part 3: Inpainting.
Read part 4: Models.

aizmin: