Local Fun with Stable Diffusion

I wanted to give Stable Diffusion a test since my experience with Midjourney went so well. I saw this write up on HackerNews a while ago and was hopeful that the little M1 MacBook could handle a AI image prompt engine. Luckily, it can. Here are my results:

Install instructions on a M1 Mac
https://replicate.com/blog/run-stable-diffusion-on-m1-mac

Space Apple

Image example from install tutorial:

Command:

python scripts/txt2img.py \
  --prompt "a red juicy apple floating in outer space, like a planet" \
  --n_samples 1 --n_iter 1 --plms

python

Time till Image: 6 min

My result:

Well, this did not turn out like the image in the tutorial, but could be an interesting album cover.

Business Cat is all business

I saw some interesting prompts on Replicate.com and wanted to see how different of an image I would get from this:

Link to example:

https://replicate.com/methexis-inc/img2prompt

Command:

python scripts/txt2img.py \
  --prompt "a cat wearing a suit and tie with green eyes, a stock photo by Hanns Katz, pexels, furry art, stockphoto, creative commons attribution, quantum wavetracing" \
  --n_samples 1 --n_iter 1 --plms

python

Time till Image: 3 min

My result:

My time to image came down a bit, but mostly due to not having to download anything prior to running. My image looks more like "In Your Face Business Cat"

Now on to some prompts I used in my Midjourney test.

Playing with Midjourney AI Art promptingComputers take’er jobs

bender from futurama fishing in the woods with a cigar

python scripts/txt2img.py \
  --prompt "bender from futurama fishing in the woods with a cigar" \
  --n_samples 1 --n_iter 1 --plms

python

Time till Image: 4 min

Scary! WTF is that?!? Do not get caught out in the woods with that.

porsche cayman electric concept car driving on a mountain road in the rain futuristic, ultra high detail, cinematic, unreal engine 5, octane render

python scripts/txt2img.py \
  --prompt "porsche cayman electric concept car driving on a mountain road in the rain futuristic, ultra high detail, cinematic, unreal engine 5, octane render" \
  --n_samples 1 --n_iter 1 --plms

python

Time till Image: 4 min

WOW! this one turned out about how I was originally intending the prompt. Nifty!

porsche 911 parked on a beach in the style of a 70s science fiction novel cover

python scripts/txt2img.py \
  --prompt "porsche 911 parked on a beach in the style of a 70s science fiction novel cover" \
  --n_samples 1 --n_iter 1 --plms

python

Time till Image: 3 min

Oh wow, you'd think this AI knows about Porsche's or something. I think a few more iterations and this could be wall art.

king of the muppets ruling the universe --ar 16:9

python scripts/txt2img.py \
  --prompt "king of the muppets ruling the universe" \
  --n_samples 1 --n_iter 1 --plms

python

Time till Image: 3 min

Well, since I can only do 512x512 images I took out the 16:9 Aspect Resolution.
This one really failed, it's like it just cut up one of the Muppets movie covers badly.

fire dragon breathing fire, black volcano background, fire particles, wallpaper -

python scripts/txt2img.py \
  --prompt "fire dragon breathing fire, black volcano background, fire particles, wallpaper" \
  --n_samples 1 --n_iter 1 --plms

python

Time till Image: 3 min

The wallpaper prompt is in Midjourney, and I did not expect it to work.

Oh my, this one turned out good, I think it will need some iterations, but I liked where it was going. I almost see a King Ghidorah in there.

A deaf mouse anthropomorphic engineer in the orbit, jet pack,
ink+concept art+line art, manga cover art + dragon ball style,
style of Doraemon, by Toriyama Akira

python scripts/txt2img.py \
  --prompt "A deaf mouse anthropomorphic engineer in the orbit, jet pack, ink+concept art+line art, manga cover art + dragon ball style, style of Doraemon, by Toriyama Akira" \
  --n_samples 1 --n_iter 1 --plms

python

Time till Image: 4 min

I triggered the NSFW filter?!? DOH!

I ran it again without the word deaf in the prompt:

Time till Image: 3 min

Ahh, that one worked, but I am sure I'd run into some sort of issue with a certain company that has a mouse as their mascot.

Final Thoughts

I love being able to run this locally on my MacBook, but find that the results are lacking a bit. Overall timing with my prompts were around 3 minutes and the Mac handles it like a champ.

This little test has me thinking that maybe you'd try both Midjourney and Stable Diffusion with an idea, then iterate on the one that gives the best result.