There are not many tutorials for generating animals. In this article, we will go through some techniques to generate
- Realistic animals
- Cute animal images
- Animal vector arts
- Fantasy animals with human bodies
- Controlling composition with ControlNet
Software
We will use AUTOMATIC1111 Stable Diffusion GUI to generate animal images. You can use this GUI on Windows, Mac, or Google Colab.
Realistic wildlife animals
Since the goal is to generate realistic photographic images, you will need to include the keyword “photo”.
The prompt should start like
photo of …
Subject
First, you will need to pick your subject(s). For example:
- Lion
- Pack of wolf
- Red panda
- Peacock
- teacup kitty
- etc…
Scene
The scene controls the background and surroundings. Because of the association effect, if you don’t add scene keywords, you will usually get the natural habitats of the wild animals.
- snow
- river
- tree
- forest
- grassland, grass field
- on a couch
Lighting
Lighting has a large effect on how the images look. Good lighting makes an image interesting.
- dark studio
- rim lighting
- sunset
- dramatic lighting
Others
Use realistic keywords similar to those for generating realistic people. For example:
- dslr
- ultra quality
- film grain
- 8K UHD
In my experience, more of these keywords are not always better. Using a few of them would already do the trick. Using too many may result in poor anatomy. I suspect many of these keywords are associated with human photos and could impair animal photos.
The following phrases can enhance the aesthetic of the wildlife images
- National Geographic Wildlife photo of the year
- The American Landscape Contest
- Wildlife photography contest
You can find more keywords in our prompt generator.
Models
You should use a model with a realistic style. For example:
- Realistic Vision
- Dreamlike Photoreal
Examples of realistic animal images
Here are some example prompts for generating realistic images. Feel free to use or remix.
Model: Realistic Vision v2.0
Prompt:
National Geographic Wildlife photo of the year, elephant trunk pointing up in new york city, night, dark studio, depth of field, trunk pointing up
Negative prompt:
deformed, disfigured, underexposed, overexposed
Model: Realistic Vision v2.0
Prompt:
National Geographic Wildlife photo of the year, red panda, evening light, sunset, rim lighting
Negative prompt:
deformed, disfigured
Model: Realistic Vision v2.0
Prompt:
Photo of (Lion:1.2) on a couch, flower in vase, dof, film grain, Fujifilm XT3, crystal clear, 8K UHD, dark studio
Model: Realistic Vision v2.0
Prompt:
National Geographic Wildlife photo of the year, siberian cat on river, evening light, sunset, rim lighting, depth of field
Negative prompt:
deformed, disfigured
Model: Realistic Vision v2.0
Prompt:
National Geographic Wildlife photo of the year, peacock flying , evening light, sunset, rim lighting, depth of field
Negative Prompt:
deformed, disfigured, underexposed, overexposed
Cute animals
Using Models
If you simply want to generate some cute animal pictures, a very simple prompt that includes the word “cute” will do the job. Pick a model to achieve a certain style.
Model: DreamShaper
Prompt:
A cute kitten
Modifying style
You can also add keywords to modify the style further with the same model.
Model: DreamShaper
Prompt:
a cute kitty, (extremely detailed CG unity 8k wallpaper), professional majestic impressionism oil painting
Negative prompt:
cartoon, 3d, disfigured, deformed easynegative
Chinese Zodiac LoRA
The Chinese Zodiac LoRA generates cute animals in a cartoon style. Use the LoRA with the sunshinemix_sunlightmixPruned model.
You can modify the prompt below to generate other animals. The suggested animals of this model are pig, bear, chook, monkey, sheep, horse, snake, dragon, bunny, tiger, cow, and rat.
Prompt:
pig, Exquisite City, (sky:1.3), (Miniature tree:1.3),Miniature object, many flowers, glowing mushrooms, (creek:1.3), lots of fruits, cute colorful animal protagonist, Firefly,meteor, Colorful cloud,Complicated background, rainbow, studio lighting, auora, rim light <lora:Chinese zodiac:1>
Negative prompt:
Void background,black background
Animal vector art
You can generate animals in different vector art styles.
Anime style
The example below uses an anime model with a simple prompt. Many anime models are fine-tuned with people, especially girls. So use the keywords people and girl in the negative prompt to get only the animal.
Model: MeinaMix
Prompt:
vector art of a horse, white background
Negative prompt:
bad art, amateur, girl, people, riding
Animal Stickers
This technique generates vector art by drawing a sticker on a white background. It does not always work, but you should get some images that can be easily cut out using Photoshop’s Select Subject function.
Model: Stable Diffusion v2.1 (768)
Prompt:
vector art of a tiger illustration stickers, ((vivid colors, colorful, pastel cute colors)), white background
low poly, tetric, mosaic, disfigured, kitsch, ugly, oversaturated, grain, low-res, Deformed, blurry, bad anatomy, disfigured, poorly drawn face, mutation, mutated, extra limb, ugly, poorly drawn hands, missing limb, blurry, floating limbs, disconnected limbs, malformed hands, blur, out of focus, long neck, long body, ugly, disgusting, poorly drawn, mutilated, mangled, old, surreal, pixel-art, black and white, childish
Animals with human clothing
You can generate animals with human bodies… it works with the Realism Engine v1. (Note that this is a v2 fine-tuned model, you will need to download the accompanying config file to use it in AUTOMATIC1111.)
Prompt:
a goat wearing a suit, dark studio
Negative prompt:
3d render, cgi, painting, drawing, cartoon, anime
Controlling poses
You can control the composition to some extent using ControlNet. You can even transfer human poses, although the animals can look strange because animal and human bodies are so different.
OpenPose
An exception is a close-up of the face. You will need to use a reference image that is a close-up human face. For example, the one below.
ControlNet: OpenPose
Model: Realistic Vision v2
Prompt:
National Geographic Wildlife photo of the year, a siberian cat, evening light, sunset, rim lighting
Negative prompt:
deformed, disfigured
Here are some close-up images of animals generated.
Canny Edge
You can transfer the composition of a wildlife photo to your image using Canny Edge.
ControlNet Setting:
- Preprocessor: Canny
- Model: Canny-fp16
- Control Weight: 0.65
- Starting control step: 0
- Ending control Step: 0.5
Prompt:
National Geographic Wildlife photo of the year, a deer, evening light, sunset, rim lighting
Negative prompt:
deformed, disfigured, woman, man, people
Reference image for ControlNet:
Generated images (various animals):
Final notes
I hope you are now familiar with some techniques that you can use to generate the animal images you want with Stable Diffusion.
It is pretty normal to get an imperfect image using any of these techniques. All you need to do is to fix some spots here or there with inpainting. So don’t give up a good image with minor defects!