Knhash

I also want to do something with Stable Diffusion

Sep 04, 2022

But I don't have a powerful GPU

Stable Diffusion is all the rage right now. If you are even remotely around the scene of AI and ML you have heard of Stable Diffusion. Rightfully so, Stable Diffusion is a big deal, majorly because they released the weights of the model. Everyone and their manager wants to get onboard this new development in the crux of Art and AI.

I too wanna. Problem is, you still need some fairly powerful GPUs to run the model. Granted the development is happening at a blistering pace (not exaggerating at all, people were able to bring down GPU requirements and get it running on M1 Macs within a couple weeks!), but I want to do something now!

I like the shoulders of giants - in fact I think some of my best work has been on top of shoulders of giants. It is comfortable to climb onto, has a very solid foundation to build on, and I can see a little bit farther, faster.

So I shall build on top of the shoulders of giants.

I built a toy pictionary game, with Stable Diffusion providing the artistic flair

If you are already excited to check it out, I call it diffAKI, and you can find it here: https://diffaki.knhash.in

There is this ultra epic site that has a bunch of artworks and prompts very neatly organized called Lexica.Art. I picked up the API from it to generate random prompt-image pairs. I show you the image, and ask you to guess the prompt. The I give you a score between 0 and 1. The scoring happens with a lighter version* of the CLIP model used in the training of Stable Diffusion (cosine similarity for the math nerds out there).

The UI is made with Streamlit and hosted on Steamlit Community Cloud. How extraordinarily cool of them to provide this for free!

I hope you have atleast as much fun playing around diffAKI as I did building it.


[*] The actual model is trained on clip-ViT-L-14 and is around 1.5GB. I couldn’t get it loaded on Streamlit Cloud so I chose the lighter clip-ViT-B-16 for running the comparisons. So in case the scores are off by a little, now you know why :)