In this, perhaps the final part of the Convolutional Art tutorial, we will be trying to create shapes using drunken walks. What's not to like!
The result of this tutorial can be found here:
https://www.openprocessing.org/sketch/587637
Setting up a grid
Javascript is notoriously bad with 2-dimensional arrays, by which I mean it does not have them, as far as I am aware. Instead we will have to use a 1-dimensional array and pretend it is 2-dimensional.
This 2-dimensional array should represent our canvas. Any x,y spot in this array will fit a x,y specific point on the canvas - or rather, when we render random points of the painting, we will look up what value is held in this array.
We will set up the array in the setup_new_generation() function, which is called whenever the program restarts:
drunkWalkWidth = floor(canvasWidth/40) ; DWW = drunkWalkWidth
drunkWalkHeight = floor(canvasHeight/30) ; DWH = drunkWalkHeight
drunkWalk = []
for(ix=0;ix<DWW;ix++)
for(iy=0;iy<DWH;iy++)
drunkWalk[ix+iy*DWW] = 0
As the name of the array so eloquently points out, we are going to be doing a drunk walk. It probably has a thousand other names, but the idea is that we are going to move a single point around randomly (drunkenly) and track its position in this array.
var nx = DWW/2
var ny = DWH/2
for(i=0;i<10000;i++)
{
nx = round(nx-1+random(2))
ny = round(ny-1+random(2))
drunkWalk[nx+ny*DWW] = 1
}
To see this in action, we are going to fast-forward a bit and, when finding the colorABCD, we will check the matching spot in the drunkWalk array - if it is one (where the drunken walk has been), colorABCD will be turned white, otherwise colorABCD will be turned black. This will show us the trail of the drunken walk:
Save for some artifacting by the bottom edge, we can quite clearly see a strange, blocky tree. I think I'll go back and change the resolution of the drunken walk.
Additionally, I will make sure that when it veers off the edge, it will come out on the other side of the screen. This can be done by adding the width and height to the coordinates each step, then modulating by the width and height, too.
There are a number of ways to change the shape of the random walk, like only changing directions every so often, but I feel like this figure is fine for our purposes.
Using the Drunken Walk
Of course, we do not want to use the drunken walk just for generating white and black parts of the image. Instead, we should add the drunken walk back into the convolutions and neurons. I can see two places to do this.
One, is that we can replace the current function of neuron[2] to instead return whether or not the drunken walk has passed through that point of the canvas.
neuron[2] = drunkWalk[floor(xcoord*DWW)+floor(ycoord*DWH)*DWW]
The other is that we can add this as another, new type of convolutional function:
case 8: //using drunken walk to return either input A or input B
neuron[i+10] = drunkWalk[floor(xcoord*DWW)+floor(ycoord*DWH)*DWW] > .5 ? inputA : inputB ; break;
I would love to show a picture of these working, but this is the best I can do:
I mean, I'm pretty sure there's a drunken walk at work somewhere here, since I bumped up the chance of choosing function 8 quite a lot, but can you really be sure? As is often the case, the convolutions are a black box, and I can only hope that things work like they should. And really, the result should be subtle. We don't want it to be too easy to see that all the paintings are made in the same way.
Foreground and background
Maybe we actually had the right idea with choosing colour directly from the drunken walk. Black and white was just a bit too exaggerated, but there must be a middle ground.
As before, we can use the drunken walk to distinguish between foreground and background. While earlier, the background was completely black and the foreground completely white, instead we can blend the background with a flat colour. Not all pictures should necessarily use this technique. This means we can add two new genes to the program:
The result of this tutorial can be found here:
https://www.openprocessing.org/sketch/587637
Setting up a grid
Javascript is notoriously bad with 2-dimensional arrays, by which I mean it does not have them, as far as I am aware. Instead we will have to use a 1-dimensional array and pretend it is 2-dimensional.
This 2-dimensional array should represent our canvas. Any x,y spot in this array will fit a x,y specific point on the canvas - or rather, when we render random points of the painting, we will look up what value is held in this array.
We will set up the array in the setup_new_generation() function, which is called whenever the program restarts:
drunkWalkWidth = floor(canvasWidth/40) ; DWW = drunkWalkWidth
drunkWalkHeight = floor(canvasHeight/30) ; DWH = drunkWalkHeight
drunkWalk = []
for(ix=0;ix<DWW;ix++)
for(iy=0;iy<DWH;iy++)
drunkWalk[ix+iy*DWW] = 0
As the name of the array so eloquently points out, we are going to be doing a drunk walk. It probably has a thousand other names, but the idea is that we are going to move a single point around randomly (drunkenly) and track its position in this array.
var nx = DWW/2
var ny = DWH/2
for(i=0;i<10000;i++)
{
nx = round(nx-1+random(2))
ny = round(ny-1+random(2))
drunkWalk[nx+ny*DWW] = 1
}
To see this in action, we are going to fast-forward a bit and, when finding the colorABCD, we will check the matching spot in the drunkWalk array - if it is one (where the drunken walk has been), colorABCD will be turned white, otherwise colorABCD will be turned black. This will show us the trail of the drunken walk:
Save for some artifacting by the bottom edge, we can quite clearly see a strange, blocky tree. I think I'll go back and change the resolution of the drunken walk.
Additionally, I will make sure that when it veers off the edge, it will come out on the other side of the screen. This can be done by adding the width and height to the coordinates each step, then modulating by the width and height, too.
There are a number of ways to change the shape of the random walk, like only changing directions every so often, but I feel like this figure is fine for our purposes.
Using the Drunken Walk
Of course, we do not want to use the drunken walk just for generating white and black parts of the image. Instead, we should add the drunken walk back into the convolutions and neurons. I can see two places to do this.
One, is that we can replace the current function of neuron[2] to instead return whether or not the drunken walk has passed through that point of the canvas.
neuron[2] = drunkWalk[floor(xcoord*DWW)+floor(ycoord*DWH)*DWW]
The other is that we can add this as another, new type of convolutional function:
case 8: //using drunken walk to return either input A or input B
neuron[i+10] = drunkWalk[floor(xcoord*DWW)+floor(ycoord*DWH)*DWW] > .5 ? inputA : inputB ; break;
I would love to show a picture of these working, but this is the best I can do:
I mean, I'm pretty sure there's a drunken walk at work somewhere here, since I bumped up the chance of choosing function 8 quite a lot, but can you really be sure? As is often the case, the convolutions are a black box, and I can only hope that things work like they should. And really, the result should be subtle. We don't want it to be too easy to see that all the paintings are made in the same way.
Foreground and background
Maybe we actually had the right idea with choosing colour directly from the drunken walk. Black and white was just a bit too exaggerated, but there must be a middle ground.
As before, we can use the drunken walk to distinguish between foreground and background. While earlier, the background was completely black and the foreground completely white, instead we can blend the background with a flat colour. Not all pictures should necessarily use this technique. This means we can add two new genes to the program:
- backgroundBlend
- backgroundCol
These two genes can be mutated like before. I am going to set backgroundBlend to a random integer between 0 and 1, and use the same script for making pigment colours as background colour. Then, when colorABCD has been found, it will be blended with backgroundBlend, depending on the drunkenwalk.
if (neuron[2] == 1) {colorABCD = lerpColor(colorABCD,backgroundCol[frameNumber],backgroundBlend[frameNumber])}
The results, here pointed out so kindly in pearly pinks:
But wait, I actually did the opposite of what I said I would! I turned the drunken walk into the background instead of the foreground! Perhaps this has its uses, though. So instead of setting backgroundBlend to a number between 0 and 1, I will set it to a number between -1 and +1 - if it is less than zero, the foreground will be the background, like in the above.
Symmetries
Right now, we are only tracking the position of the dot in the drunken walk, which will create lopsided, assymetric shapes. Which is great! Sometimes at least. But maybe we might like some symmetry too.
When we set up our drunkenwalk, we can also set up a couple extra variables, xmirror and ymirror, which will be either -1 or 1. And then, after tracing the first point:
drunkWalk[nx+ny*DWW] = 1
We can trace its mirror buddy:
drunkWalk[nx*xmirror+ny*DWW*ymirror+DWW*(xmirror<1)+DWW*DWH*(ymirror<1)] = 1
When both xmirror and ymirror are 1, the same position will be set to 1 twice. However, if xmirror is -1, the figure will be mirrored horizontally - ymirror, vertically, and with both, diagonally. To make sure not all of the frame is taken up by the random walk, we can halve the amount of steps, down to 5000.
Here we see diagonal symmetry carving out a strange symbol in the centre, as well as the four corners. Looking it over, I think it will be safe to reduce the drunkenwalk down to 3000 steps instead. More than half the picture is taken up by it! Or perhaps a random amount of steps could be picked. The options are limitless.
Reintroducing digital colours
Implementing the drunken walk turned out to be much easier than expected, so we can spend the rest of this tutorial messing up the colours we have so simply relied on for so long.
Way back in the first tutorial, we looked at two sorts of colours - digital colours with three independent colour channels - and pigment colours, that we ended up settling on. How about we make both colours - the pigment mix colorABCD, and a digital HSB color - and then let the neural net pick control when which of them should be used?
colorMode(HSB,1)
colorDigital = color(neuron[27],neuron[28],neuron[29])
colorMode(RGB,256)
colorABCD = lerpColor(colorABCD,colorDigital,sq(neuron[26]))
Using the square of neuron[26] to control the amount of digital colour used will ensure that it will only come through at a few places on the canvas. And who doesn't want to add a bit of rainbow colouring here and there?
Well, then. That's nice, isn't it? And to see much more of this, you can go to:
Where the program and all the code can be found.
That's it for now
This might be the last part of the tutorial, so all I can say for now is good luck! I hope the approach or some of the methods have inspired you for doing some generative art on your own. As always, I am more than happy to answer any questions. No matter how in-depth I try to make the tutorial, there are still so many more depths left unexplored.
The whole project has been, er, a bit difficult. Not so much programming, not so much writing, mostly doing both at once. Programming something like this requires a lot of trial-and-error, a lot of tinkering, and it is really difficult to have to write something, then feel limited in changing the process later on. It makes the process of programming a lot more "think-before-you-act".
Still, it hasn't been that bad. The reason it took me so long is mostly because of procrastinating and working on other projects, as well as wanting to space out posts.
My original goal was actually to make a better art generator than the one I currently have in my personal .gml project. This did not quite work out, though this art generator really is quite competent on its own. I had the idea that limiting the neurons to values between 0 and 1 would make things easier. It did! My own generator is incredibly janky and unpredictable. But perhaps that jank and unpredictability was actually its boon.
Some WIP images and extra paintings from this can be found on my twitter:
Comments
Post a Comment