scenery

large scales. In this way it builds a mathematical abstraction of what an object looks like and forms and then strengthens a link between that visual and the token for the text that describes it. It’s capable enough to also make associations between what we would call aesthetics and tokens for the text annotations. So, for example, if it sees enough photos of dog, crude crayon drawings of dogs, and low poly 3d renders of dogs, and the data is properly annotated, it will extract a mathematical representation of photos and associate that with “realism”, the crayon scribbles with “child’s drawing”, and the render with “video game engine”.

Okay so far so good… what does any of this have to do with anything?

Well the training data fed into this implementation of stable diffusion is not just a random image scraping of the whole internet. For one, they wanted to exclude anything naughty, ie pr0n (don’t want my comment hidden for using the real word), as well as legitimate sick stuff, like

^^ naw son it’s cooler than kinda cool lol - keeping the whole scene unfocused like this is tricky!

Why? Well because the only way the model knows what X Y or Z looks like is through training - a process of feeding a model a seriously truly epically gigantic dataset containing two or more different types of data, with each datapoint containing one or more tokens (items that can be mathematically correlated).

The deep learning model through trial and error builds up an associative map between the various tokens, creating links between them.

For visual discrimination or synthesis (quite often synthesis is mostly discrimination in reverse) the training dataset consists of a monumentally large collection of images, each with annotations describing what is in the image. The annotations are tokens in this case, but the visual part of the model breaks the pictures down and learns relationships between the content of pixels at immediate scales (adjacent neighbors), medium scales, and

kinda cool