Stable Diffusion is a state-of-the-art text-to-image generator that can create realistic and diverse images from natural language prompts. It is based on a latent diffusion model that learns to synthesize images by minimizing an energy function that measures how well the image matches the text description. Stable Diffusion has been used for various creative and artistic purposes, such as generating logos, portraits, landscapes, cartoons, and more.
However, to use Stable Diffusion effectively, you need to understand the functions of the menu that are available in different web user interfaces (WebUIs) that provide access to the model. In this article, we will explain the functions of the menu in Stable Diffusion, using the AUTOMATIC1111 WebUI as an example. We will cover the following topics:
How to import new models and select different models
One of the advantages of Stable Diffusion is that it can handle a variety of models that are trained on different datasets, domains, and resolutions. For example, there are models that are specialized in generating anime, fantasy, or realistic images. To use these models, you need to import them into the WebUI and select them from the menu.
To import a new model, you need to download a compatible model file with a .ckpt or .safetensors file extension. If both versions are available, it is advised to go with the safetensors one. Once you have downloaded your model, you need to put it in the stable-diffusion-webui/models directory. Then, you need to restart the WebUI and refresh the browser. You should see the new model in the model selection menu.
To select a different model, you simply need to click on the model name in the menu and choose the one you want to use. The WebUI will automatically load the model and display its information, such as the resolution, the text encoder, and the sampling method. You can also see the model card that provides more details about the model, such as the training data, the parameters, and the examples.
How to write a good prompt and use weighted and negative prompts
A prompt is a text input that describes what you want to see in the image. For example, if you want to generate an image of a cat, you can write “a cat” as your prompt. However, this prompt is too vague and can produce many different images of cats, depending on the model’s interpretation. To get more specific and consistent results, you need to write a more detailed and specific prompt, such as “a cute grey cat with blue eyes sitting on a sofa”.
A prompt is important because it guides the model to generate the image that matches your intention. A good prompt should be clear, concise, and relevant to the image you want to create. A bad prompt can lead to poor or unexpected results, such as images that are blurry, distorted, or irrelevant to the prompt.
To write a good prompt, you can follow some general principles and techniques, such as:
- Be as specific as you can. Stable Diffusion tends to thrive on specific prompts, especially when compared to other text-to-image generators. You need to tell it exactly what you want, using descriptive words and phrases that narrow down the image. For example, instead of writing “a landscape”, you can write “a snowy mountain landscape with a lake and a cabin”.
- Use weighted prompts to emphasize, deemphasize, or avoid certain elements. Weighted prompts are a syntax that allows you to assign different weights to different keywords or phrases in your prompt, using parentheses and brackets. For example, you can write “(a cat) [0.8] (a dog) [0.2]” to generate an image that is more likely to contain a cat than a dog. You can also use negative weights to reduce or eliminate certain elements, such as “(a cat) [-1] (a dog) 1” to generate an image that does not contain a cat, but only a dog. Weighted prompts are a powerful way to control the image generation process and fine-tune your results12.
- Use negative prompts to remove unwanted elements. Negative prompts are another syntax that allows you to specify what you don’t want in the image, using the word “not” before the keywords or phrases. For example, you can write “a cat not on a sofa” to generate an image of a cat that is not sitting on a sofa. Negative prompts are useful to steer the image away from common or default elements that the model may generate otherwise3.
- Use celebrity names, artist names, or website names to influence the appearance, style, or quality of the image. Stable Diffusion has a large and diverse knowledge base that includes many famous names and entities. You can use these names as keywords or phrases in your prompt to affect the image generation in various ways. For example, you can use celebrity names to control the face, pose, or expression of the human subject, such as “Taylor Swift smiling”. You can use artist names to control the style, mood, or theme of the image, such as “Van Gogh’s Starry Night”. You can use website names to control the resolution, format, or quality of the image, such as “Pinterest quality”. These names have a strong association effect that can help you achieve your desired results45.
- Use camera angles, distance, and lighting to adjust the perspective, scale, and illumination of the image. Stable Diffusion enables the control of camera angles, distance, and lighting using prompts, by using keywords or phrases that indicate the direction, position, or intensity of the camera or the light source. For example, you can use “top view” or “bottom view” to change the angle of the camera, “close up” or “far away” to change the distance of the camera, or “bright” or “dark” to change the lighting of the image. These prompts can help you create more dynamic and realistic images67 .
To write a prompt, you simply need to type it in the prompt box in the menu. You can also use the preset prompts that are available for some models, or the random prompt button that generates a random prompt for you. You can also edit the prompt after generating an image, and the WebUI will automatically update the image accordingly.
How to choose a sampling method and adjust the sampling steps
Stable Diffusion uses a latent diffusion process to generate images from text prompts. This process involves sampling from a probability distribution that is conditioned on the text and the image resolution. There are different sampling methods that can affect the quality, diversity, and speed of the image generation. The WebUI allows you to choose from four sampling methods:
- Automatic1111: This is the default and recommended sampling method. It uses a dynamic sampling schedule that adapts to the text prompt and the model. It produces high-quality and diverse images in a reasonable time. It also supports weighted and negative prompts, as well as seeds and style selection.
- Linear: This is a simple and fast sampling method. It uses a fixed sampling schedule that linearly increases the noise level from 0 to 1. It produces low-quality and low-diversity images, but it can be useful for testing or debugging purposes. It does not support weighted and negative prompts, seeds, or style selection.
- Geometric: This is a more advanced and slower sampling method. It uses a fixed sampling schedule that geometrically increases the noise level from 0 to 1. It produces high-quality and high-diversity images, but it can be very slow and unstable. It supports weighted and negative prompts, as well as seeds and style selection.
- Custom: This is a flexible and experimental sampling method. It allows you to customize the sampling schedule by specifying the noise level and the number of steps for each stage. It can produce various results depending on the settings, but it can also be very slow and unstable. It supports weighted and negative prompts, as well as seeds and style selection.
To choose a sampling method, you simply need to click on the sampling method name in the menu and select the one you want to use. The WebUI will automatically load the sampling method and display its information, such as the number of steps, the noise level, and the sampling time.
To adjust the sampling steps, you can use the slider or the input box in the menu to change the number of steps for each stage. The number of steps determines how many times the model updates the image with the text prompt. The more steps, the more refined the image, but also the more time it takes. The optimal number of steps depends on the model, the prompt, and the sampling method. You can experiment with different values to find the best balance between quality and speed.
How to change the image size and the CFG scale
Stable Diffusion can generate images at different resolutions, depending on the model and the settings. The WebUI allows you to change the image size and the CFG scale from the menu.
The image size determines the output resolution of the image. It can be set to 64x64, 128x128, 256x256, or 512x512 pixels. The higher the image size, the more detailed the image, but also the more time and memory it takes. The image size should match the resolution of the model, otherwise the image may look distorted or pixelated.
The CFG scale determines the size of the text encoder’s output. It can be set to 1, 2, 4, or 8. The higher the CFG scale, the more information the text encoder can provide to the image generator, but also the more time and memory it takes. The CFG scale should match the scale of the model, otherwise the image may look blurry or noisy.
To change the image size and the CFG scale, you simply need to click on the image size or the CFG scale name in the menu and select the one you want to use. The WebUI will automatically update the settings and display the image size and the CFG scale in the menu. You can also see the estimated VRAM usage and the sampling time for each setting. You can experiment with different values to find the best balance between quality and speed.
How to use batch count and batch size
Stable Diffusion can generate multiple images at once, using a feature called batch generation. This feature allows you to increase the diversity and the efficiency of the image generation process. The WebUI allows you to control the batch count and the batch size from the menu.
The batch count determines how many images are generated for each prompt. It can be set to 1, 2, 4, or 8. The higher the batch count, the more diverse the images, but also the more time and memory it takes. The batch count should match the number of images you want to see for each prompt.
The batch size determines how many images are generated in parallel. It can be set to 1, 2, 4, or 8. The higher the batch size, the faster the images, but also the more memory it takes. The batch size should match the capacity of your GPU and the resolution of the model.
To change the batch count and the batch size, you simply need to click on the batch count or the batch size name in the menu and select the one you want to use. The WebUI will automatically update the settings and display the batch count and the batch size in the menu. You can also see the estimated VRAM usage and the sampling time for each setting. You can experiment with different values to find the best balance between diversity and speed.
How to use seeds and style selection
Stable Diffusion can generate images with different styles and variations, depending on the random seed and the style selection. The WebUI allows you to control the seed and the style selection from the menu.
The seed determines the initial state of the random number generator that is used by the model. It can be set to any integer value between 0 and 2^32-1. The same seed will always produce the same image for the same prompt and model. The different seed will produce different images for the same prompt and model. The seed can be used to reproduce or change the results.
The style selection determines the latent vector that is used by the model to influence the style and the appearance of the image. It can be set to any integer value between 0 and 1023. The same style selection will always produce the same style for the same prompt and model. The different style selection will produce different styles for the same prompt and model. The style selection can be used to fine-tune or explore the results.
To change the seed and the style selection, you can use the slider or the input box in the menu to change the value for each setting. The WebUI will automatically update the settings and display the seed and the style selection in the menu. You can also use the random seed and the random style buttons to generate random values for each setting. You can experiment with different values to find the best combination of seed and style for your prompt.
How to use checkpoint merging and script selection
Stable Diffusion can generate images using multiple models that are merged together, using a feature called checkpoint merging. This feature allows you to combine the strengths and the features of different models, such as resolution, domain, and quality. The WebUI allows you to use checkpoint merging and script selection from the menu.
Checkpoint merging requires two or more compatible models that have the same text encoder and the same CFG scale. The models are merged by using a script that specifies how to combine the checkpoints of each model. The script can be written in Python or JSON format, and it can define various parameters, such as the weights, the layers, and the sampling methods of each model.
Script selection allows you to choose from different scripts that are available for some models, or to write your own custom script. The scripts can be used to create different effects, such as blending, mixing, or enhancing the images. The scripts can also be used to create new models that are not available otherwise, such as high-resolution or cross-domain models.
To use checkpoint merging and script selection, you need to import two or more compatible models into the WebUI and select them from the menu. Then, you need to click on the script selection name in the menu and choose the script you want to use. The WebUI will automatically load the script and display its information, such as the name, the description, and the parameters. You can also edit the script or write your own custom script in the script box in the menu. You can experiment with different scripts and parameters to find the best combination of models for your prompt.
How to use image upscaling and img2img functionality
Stable Diffusion can generate images at higher resolutions than the original model, using a feature called image upscaling. This feature allows you to increase the quality and the detail of the images, using a super-resolution model that is trained on the same dataset as the original model. The WebUI allows you to use image upscaling from the menu.
Image upscaling requires a compatible super-resolution model that has the same resolution and the same domain as the original model. The super-resolution model is applied to the output of the original model, and it enhances the image by adding more pixels and more details. The result is an image that has a higher resolution and a higher quality than the original image.
To use image upscaling, you need to import a compatible super-resolution model into the WebUI and select it from the menu. Then, you need to click on the image upscaling name in the menu and choose the resolution you want to use. The WebUI will automatically load the super-resolution model and display its information, such as the resolution, the text encoder, and the sampling method. You can also see the estimated VRAM usage and the sampling time for each resolution. You can experiment with different resolutions to find the best balance between quality and speed.
Stable Diffusion can also generate images from existing images, using a feature called img2img functionality. This feature allows you to modify or transform the images, using a text prompt that describes the changes you want to make. The WebUI allows you to use img2img functionality from the menu.
Img2img functionality requires an image file that is compatible with the model and the prompt. The image file can be uploaded from your computer or from the web, and it should have the same resolution and the same domain as the model and the prompt. The image file is used as the input of the model, and it is modified or transformed by the text prompt. The result is an image that has the same resolution and the same domain as the original image, but with different features or styles.
To use img2img functionality, you need to click on the img2img button in the menu and choose the source of the image file. Then, you need to upload the image file from your computer or from the web, and it will be displayed in the menu. You can also see the image size and the image name in the menu. Then, you need to write a prompt that describes the changes you want to make to the image, and click on the generate button. The WebUI will automatically apply the prompt to the image and display the result. You can experiment with different prompts and images to find the best combination of img2img functionality for your purpose.
Conclusion
Stable Diffusion is an amazing text-to-image generator that can create realistic and diverse images from natural language prompts. However, to use Stable Diffusion effectively, you need to understand the functions of the menu that are available in different WebUIs that provide access to the model. In this article, we have explained the functions of the menu in Stable Diffusion, using the AUTOMATIC1111 WebUI as an example. We have covered the following topics:
- How to import new models and select different models
- How to write a good prompt and use weighted and negative prompts
- How to choose a sampling method and adjust the sampling steps
- How to change the image size and the CFG scale
- How to use batch count and batch size
- How to use seeds and style selection
- How to use checkpoint merging and script selection
- How to use image upscaling and img2img functionality
We hope you have learned something useful and enjoyed this article. Thank you for reading! 😊
Post a Comment