How to use prompt correctly in Stable Diffusion

Stable Diffusion is a powerful text-to-image generator that can create realistic and diverse images from natural language prompts. However, writing effective prompts is not always easy, as it requires a good understanding of the model’s capabilities, limitations, and preferences. In this article, we will show you how to use prompt correctly in Stable Diffusion, with some tips and tricks to improve your results.

What is prompt and why is it important?

A prompt is a text input that describes what you want to see in the image. For example, if you want to generate an image of a cat, you can write “a cat” as your prompt. However, this prompt is too vague and can produce many different images of cats, depending on the model’s interpretation. To get more specific and consistent results, you need to write a more detailed and specific prompt, such as “a cute grey cat with blue eyes sitting on a sofa”.

A prompt is important because it guides the model to generate the image that matches your intention. A good prompt should be clear, concise, and relevant to the image you want to create. A bad prompt can lead to poor or unexpected results, such as images that are blurry, distorted, or irrelevant to the prompt.

How to write a good prompt for Stable Diffusion?

Stable Diffusion can handle a variety of prompts, from simple keywords to complex sentences. However, there are some general principles and techniques that can help you write a good prompt for Stable Diffusion. Here are some of them:

• Be as specific as you can. Stable Diffusion tends to thrive on specific prompts, especially when compared to other text-to-image generators. You need to tell it exactly what you want, using descriptive words and phrases that narrow down the image. For example, instead of writing “a landscape”, you can write “a snowy mountain landscape with a lake and a cabin”.
• Use weighted prompts to emphasize, deemphasize, or avoid certain elements. Weighted prompts are a syntax that allows you to assign different weights to different keywords or phrases in your prompt, using parentheses and brackets. For example, you can write “(a cat) [0.8] (a dog) [0.2]” to generate an image that is more likely to contain a cat than a dog. You can also use negative weights to reduce or eliminate certain elements, such as “(a cat) [-1] (a dog) 1” to generate an image that does not contain a cat, but only a dog. Weighted prompts are a powerful way to control the image generation process and fine-tune your results¹².
• Use negative prompts to remove unwanted elements. Negative prompts are another syntax that allows you to specify what you don’t want in the image, using the word “not” before the keywords or phrases. For example, you can write “a cat not on a sofa” to generate an image of a cat that is not sitting on a sofa. Negative prompts are useful to steer the image away from common or default elements that the model may generate otherwise³.
• Use celebrity names, artist names, or website names to influence the appearance, style, or quality of the image. Stable Diffusion has a large and diverse knowledge base that includes many famous names and entities. You can use these names as keywords or phrases in your prompt to affect the image generation in various ways. For example, you can use celebrity names to control the face, pose, or expression of the human subject, such as “Taylor Swift smiling”. You can use artist names to control the style, mood, or theme of the image, such as “Van Gogh’s Starry Night”. You can use website names to control the resolution, format, or quality of the image, such as “Pinterest quality”. These names have a strong association effect that can help you achieve your desired results⁴⁵.
• Use camera angles, distance, and lighting to adjust the perspective, scale, and illumination of the image. Stable Diffusion enables the control of camera angles, distance, and lighting using prompts, by using keywords or phrases that indicate the direction, position, or intensity of the camera or the light source. For example, you can use “top view” or “bottom view” to change the angle of the camera, “close up” or “far away” to change the distance of the camera, or “bright” or “dark” to change the lighting of the image. These prompts can help you create more dynamic and realistic images⁶⁷⁸.

Examples of good prompts for Stable Diffusion

To illustrate how to use prompt correctly in Stable Diffusion, here are some examples of good prompts and the images they generate. We will use the v1.5 base model and the Automatic1111 sampler for these examples.

• A realistic portrait of a young woman with long blonde hair and blue eyes, wearing a red dress and a pearl necklace, smiling and looking at the camera

This prompt is very specific and descriptive, using keywords and phrases that define the appearance, clothing, expression, and pose of the woman. The result is a realistic and consistent portrait that matches the prompt.

!A realistic portrait of a young woman with long blonde hair and blue eyes, wearing a red dress and a pearl necklace, smiling and looking at the camera

• A fantasy landscape of a floating island with a castle, a waterfall, and a rainbow, in the style of Studio Ghibli

This prompt is creative and imaginative, using keywords and phrases that describe the elements and features of the fantasy landscape. The style keyword indicates the artistic influence of the image, which is Studio Ghibli, a famous Japanese animation studio. The result is a fantasy and whimsical landscape that resembles the style of Studio Ghibli.

!A fantasy landscape of a floating island with a castle, a waterfall, and a rainbow, in the style of Studio Ghibli

• A surreal image of a giant octopus attacking a city, with a stormy sky and lightning, not in the style of H.P. Lovecraft

This prompt is bizarre and unusual, using keywords and phrases that describe the scene and the mood of the image. The negative style keyword indicates the exclusion of a certain influence, which is H.P. Lovecraft, a famous horror writer who often featured giant octopuses in his stories. The result is a surreal and terrifying image that does not resemble the style of H.P. Lovecraft.

!A surreal image of a giant octopus attacking a city, with a stormy sky and lightning, not in the style of H.P. Lovecraft

• A cute cartoon of a cat and a dog wearing Christmas hats and scarves, sitting on a couch and watching TV, with a fireplace and a Christmas tree in the background

This prompt is cute and festive, using keywords and phrases that describe the characters, the accessories, the setting, and the theme of the image. The result is a cute and cozy cartoon that depicts a cat and a dog enjoying a Christmas night.

!A cute cartoon of a cat and a dog wearing Christmas hats and scarves, sitting on a couch and watching TV, with a fireplace and a Christmas tree in the background

• A high-quality logo for a company called “Stable Diffusion”, with a blue and green color scheme and a modern font

This prompt is simple and professional, using keywords and phrases that describe the name, the colors, and the font of the logo. The quality keyword indicates the resolution and the format of the image, which is high and suitable for a logo. The result is a sleek and elegant logo that represents the company name and the color scheme.

!A high-quality logo for a company called “Stable Diffusion”, with a blue and green color scheme and a modern font

Conclusion

Stable Diffusion is a remarkable text-to-image generator that can create amazing images from natural language prompts. However, writing effective prompts is not always easy, as it requires a good understanding of the model’s capabilities, limitations, and preferences. In this article, we have shown you how to use prompt correctly in Stable Diffusion, with some tips and tricks to improve your results. We hope you have learned something useful and enjoyed this article. Thank you for reading! 😊