Comparing Stable Diffusion and DALL-E 3 for AI Image Generation

Table of Contents

• Introducing Stable Diffusion and DALL-E 3
• Generating Images from AI-Written Prompts
• Using Plugins to Generate Prompts
• Conclusion and Key Takeaways

Introducing Stable Diffusion and DALL-E 3

Stable Diffusion and DALL-E 3 are two of the most advanced AI image generation models available today. They utilize deep learning techniques to generate highly realistic and creative images simply from text prompts. In this post, we'll provide an overview of their capabilities and interfaces, and then demonstrate generating images using prompts written by AI assistant ChatGPT.

Stable Diffusion was created by Anthropic and is based on a text-to-image diffusion model. It can generate 512x512 pixel images and has been trained on a massive dataset of image-text pairs from the internet. DALL-E 3 was created by OpenAI and is the latest iteration of their DALL-E model. It can generate 1024x1024 images with exceptional quality and consistency.

Overview of Capabilities

Both Stable Diffusion and DALL-E 3 allow generating images from text prompts with a high degree of photorealism and creativity. They support generating images of people, animals, objects, scenes and more. Their interfaces allow tweaking parameters like image size, number of images generated, image coherence, etc. A key advantage of both models is their ability to understand nuances in language and generate images accordingly. They can handle prompts with multiple details, attributes, styles and contextual relationships. This makes them extremely flexible tools for illustrating concepts, ideas and stories.

User Interface and Settings

Stable Diffusion has both a command line interface as well as user-friendly web UIs like Automatic1111. This allows easily generating images through text prompts and adjusting sampling parameters. DALL-E 3 is accessible through OpenAI's web interface which also provides ample control over image size, style and other attributes. Both interfaces make iterative improvements convenient by allowing prompt editing and fast re-generation. Overall, the interfaces contain similar core parameters but DALL-E 3 has some additional advanced features like image outpainting.

Generating Images from AI-Written Prompts

To demonstrate the image generation capabilities of Stable Diffusion and DALL-E 3, we'll use the AI assistant ChatGPT to provide imaginative text prompts. Then we'll feed these prompts into both models to compare the quality of resulting images.

For each of the following prompt sets, we provided ChatGPT with a request to generate random, unusual and creative prompts. We copied these prompts into the Stable Diffusion and DALL-E 3 interfaces with default settings, and generated images. Let's examine some examples!

Prompt Set 1: Kiwi Made of Gears

For the first prompt, ChatGPT described: "A mechanically kiwi bird made of intricate gears and cogs". Stable Diffusion generated a bike-like image that partially captured the idea. However, DALL-E 3 created an incredibly detailed metallic kiwi with visible gear components - perfectly matching the prompt! This example illustrates DALL-E 3's greater proficiency for detailed and unusual prompts. The gears are neatly integrated into the kiwi form and properly layered.

Prompt Set 2: Cosmic Sushi Conveyor Belt

The second prompt was: "A sushi conveyor belt that winds its way through different planets, stars and nebula with alien species picking up their favorite cosmic sushi rolls." Stable Diffusion produced a convoluted image with some sushi elements. But DALL-E 3 generated a creative scene with a space conveyor belt, aliens, and cosmic backgrounds. The vivid details like the sushi dishes and alien species follow the prompt nicely.

Prompt Set 3: Underwater Merman Cafe

ChatGPT suggested: "A merman barista in a bubble cafe, an underwater cafe where merpeople sip on bubble lattes. The barista, a dashing merman, crafts intricate latte art using colorful coral and seaweed." Stable Diffusion partially picked up on the mermaid/underwater aspects but missed key details. DALL-E 3 perfectly depicted the underwater bubble cafe with a merman barista creating latte art as described. The lighting and background also nicely match the scene.

Prompt Set 4: Haunted Typewriter

For the fourth prompt, we received: "An old, rusty typewriter in a dimly lit attic with ghostly hands emerging from the shadows typing out verses of eerie poetry that float in the air like wisps of smoke." Again, Stable Diffusion only generated a basic typewriter image while DALL-E 3 created an atmospheric haunted typewriter scene with ghostly hands and smoke-like poetry, just as the prompt instructed.

Prompt Set 5: Formula 1 in Space

The final prompt from ChatGPT was: "A Formula 1 car racing on a track that spirals around an arglass in the vastness of space. The pit crew consists of clockwork beings replacing tires with liquid nitrogen." Stable Diffusion produced a muddled image with vague Formula 1 and galaxy elements. DALL-E 3 perfectly depicted a Formula 1 race in space with the described scene details like the arglass track and clockwork pit crew.

Using Plugins to Generate Prompts

In addition to writing prompts manually, there are also tools that can help generate interesting prompts algorithmically. As an example, we used the Imagen mixer plugin which leverages the power of ChatGPT to provide unique prompt ideas.

We had it suggest two India-related prompt concepts, which we then used to generate images with Stable Diffusion and DALL-E 3. This demonstrates how plugins and AI assistants can provide a continual stream of creative ideas to feed into the image generators.

Prompt Set 6: Cricket Match Between Gods

The plugin provided the prompt: "A cricket match between Indian gods and mythological creatures. The Indian gods look alike to their traditional depictions and the mythological creatures are fantastical." Stable Diffusion struggled to depict this properly, generating muddled figures. DALL-E 3 nicely illustrated a cricket match between identifiable Indian gods and mythological creatures as described.

Prompt Set 7: Bollywood Dance on the Moon

The second plugin-generated prompt was: "A Bollywood dance sequence on the moon with Taj Mahal visible on the surface of the moon and Earth in the background." Again, Stable Diffusion produced an unclear image while DALL-E 3 vividly depicted Bollywood dancers on the moon with the Taj Mahal and Earth visible as requested.

Conclusion and Key Takeaways

From these examples, we can see that DALL-E 3 consistently produces higher fidelity images that better match unique, detailed prompts. It handles unusual concepts, contextual relationships and multiple elements more effectively than Stable Diffusion in its current form.

However, both models are extremely capable and useful creative tools. With the right prompts and parameter tweaking, Stable Diffusion can also generate impressive results. The interfaces allow quickly iterating to improve outputs.

Going forward, generative AI will enable generating limitless high-quality images from text. Assistants like ChatGPT can provide creative prompt ideas at scale. Overall these models open enormous possibilities for illustrations, content creation, art and more!

FAQ

Q: What is Stable Diffusion used for?
A: Stable Diffusion is an AI system used to generate images from text descriptions and prompts.

Q: How does DALL-E 3 compare to Stable Diffusion?
A: DALL-E 3 produces more detailed and accurate images from prompts compared to Stable Diffusion.

Q: What plugins were used in the video?
A: The video demonstrated using the Mixen plugin to generate random prompts for the AI image generators.