Convolutional Art - Medium Matters

Coming up with a method to generate interesting colour-schemes, shapes, contrasts and such can be a challenge in itself, but I will argue that perhaps more than colour, the medium matters.


These four pictures have the same underlying colours; the only difference is how they are rendered; how they mimick a certain medium.

In the beginning of my work with the Convolutional Art generator, what I did was a pixel render, which would just loop through every pixel of the image and colour it according to some arcane process. One problem is that it creates relatively flat images that look much more digital than like concrete objects. The other is that it might make things look unappealing.

I think it has to do with a much too high level of detail, but detail of a most unnatural kind. There is at once both chaos and smooth gradients, and if you look for long enough, you can find what look like geometric lines and graphs explaining most of it. I absolutely hate it.



 Here, the pixel rendering has been replaced with what I term the "fluffy cloud" method. The algorithm jumps to random spots all over the canvas, finds the colour like before, and draws a soft splotch of paint.

The splotch in question looks like this, and was inspired by this post by the excellent Sighack.
If we were to use a completely fluffy texture, like that of a circle, the painting would only be even more flat. With this, well, it's not quite enough.

For it to become something more aesthetic, the detail has to be downsampled, as we have done with the fluffy clouds, but some places in the image might best retain their high level of detail.


The difference here is quite subtle, but if you compare the two images, you will see that indeed, some places have more solid splotches of paint while others stay blurry. This is inspired by the well-known effect of focus when taking pictures, that sharp edges next to blurry ones look so, so, so incredibly crisp. The sharp and blurry parts of the image guide the eyes and allow us to take in some parts of the image above others.

How to determine what should be brought forth and what should stay blurry? Well, with my method, I have several neural layers just littering the place, so I use one of them. For your purposes, maybe you can pick a fixed compostion - focus on the centre, or below the median - or good old Perlin noise.


 Still, the picture seems rather motionless. Very few artistic styles actually just paint dots of colour spread over the canvas, so why should we? Here, longer brush-strokes are made by "dragging" the colour across the canvas for a certain distance. Instead of picking random directions, Perlin flow-fields are used.

Again, I do not just let all parts of the image be equally brushy or equally pointillistic. Instead, I once again use a spare neural layer to dertermine where the picture should be in motion, and where it should not be. Since the neural layers do not have a uniform average, some pictures will have a generally short brush-stroke, while others have generally long ones.


I hope this was a good introduction to some methods one can use, on top of prior procedural generation methods, to create aesthetically pleasing pieces. Finally, I will show a few more random compositions like the one above. The top-left picture will be a pixel render. The top-right will be equal-sized fluffy clouds. The bottom-left will be fluffy clouds of varying sizes. And finally, the bottom-right will add in the longer brush-strokes.


It might seem like a waste to simply discard all that extra detail in the first picture, but it really is not. Whenever a paint-splotch is rendered, it might land in any of the different colours in close-proximity to each-other. Thus the incredible amount of lines in the first picture translates into the dense colour variation in the later, more soft renderings.



Here we see that the neural layers have deemed that this specific picture should be even more blurry than before we gave it the choice of blurryness. Still some of the flatness is combatted in the end with the brush strokes and their added texture.



Here we see that the different levels of blurriness are very important for this piece, having some rings seem to be in the front of the image, while the rest is in the back. On the other hand, the neural layer seems to deem long brush-strokes unnecessary.

Finally, what a glitchy mess the first rendering is. Luckily, adding a medium and structure to the piece lets the final piece be somewhere be at least mediocre.