After I made the procedural dungeon generator, I was stumped by the question of what to do with it. For some reason, the first idea that came to mind was that dungeons need adventurers - and just like I did not want to build the dungeons myself, neither did I want to play them myself. Instead, I wanted a neural network to control the adventurer:
Or rather, to get a lot of neural networks to control a lot of adventurers, then see which ones did best, and let them proliferate and evolve:
To kind of lacklustre results. Here, the less succesful iterations are coloured dark and translucent, while the best ones are bright and opaque. Sure, some of the adventurers do quite well, exploring the dungeon, picking up coins and killing slimes - but there are a lot of shades walking around pointlessly. And this lasted despite hours of training.
I think the biggest problem is the random dungeons. Neural networks do best when dealing with input they have seen before, but here, every generation was sent into a completely new and different dungeon. The strategies that were succesful in one dungeon might not be sucessful in the next.
Of course, the point of neural networks is to be able to extrapolate beyond the initial training set - but still, one usually lets the network start training on a consistent set first.
If instead, I let the neural network run on the same dungeon over and over, this is the result:
The first generation goes about as poorly as one might expect. A lot of adventureres get stuck in geometry and keep swinging their swords for no reason. Since there is a coin just north of the spawnpoint, though further north is a cul de sac, those who venture north are actually some of the more succesful iterations.
By generation five, though a lot of adventurers still go north for a coin that is no longer there (I guess that's a bug, oops), the more succesful iterations instead explore further into the dungeon. Though coming close with slimes is quite dangerous, it can also be quite rewarding - slaying a slime nets a hefty amount of points. Unfortunately, while showing all adventurers at once, you can't see which slimes have been slain by which adventurers - just believe that if an adventurer passes through a slime without dying, they must already have killed it.
Finally, by generation 50, most adventurers have found out that the most advantageous route is to go east, slime-hunting. Though they are getting a lot further than the original iterations, it is not a gracious sight. Lots of sword-swinging. If I had let the animation go on, it would show them continously stumbling around the room, never realising that it might be time to leave once there are no more slimes.
Overall, this has been a very tedious and pretty disappointing project. I could spend a long time criticising the method, but I actually think the biggest problem is that I made the game too difficult.
Later on, I made it so that I could control one of the adventurers, and oh my god is it difficult. The tile-based movement, the unpredictable slimes and the unwieldy sword that can only hit in one of four directions. Again and again, I would stand diagonally to a slime in a terrible stale-mate. The second I left to be able to strike it, it would move too, winning the game by either escaping back to a new stale-mate or by moving into me, enveloping me and winning.
Some times, common sense might be just as important as the finer details of how I decided to train the neural network. Don't teach a computer to walk if you cannot crawl yourself.
Or rather, to get a lot of neural networks to control a lot of adventurers, then see which ones did best, and let them proliferate and evolve:
To kind of lacklustre results. Here, the less succesful iterations are coloured dark and translucent, while the best ones are bright and opaque. Sure, some of the adventurers do quite well, exploring the dungeon, picking up coins and killing slimes - but there are a lot of shades walking around pointlessly. And this lasted despite hours of training.
I think the biggest problem is the random dungeons. Neural networks do best when dealing with input they have seen before, but here, every generation was sent into a completely new and different dungeon. The strategies that were succesful in one dungeon might not be sucessful in the next.
Of course, the point of neural networks is to be able to extrapolate beyond the initial training set - but still, one usually lets the network start training on a consistent set first.
If instead, I let the neural network run on the same dungeon over and over, this is the result:
The first generation goes about as poorly as one might expect. A lot of adventureres get stuck in geometry and keep swinging their swords for no reason. Since there is a coin just north of the spawnpoint, though further north is a cul de sac, those who venture north are actually some of the more succesful iterations.
By generation five, though a lot of adventurers still go north for a coin that is no longer there (I guess that's a bug, oops), the more succesful iterations instead explore further into the dungeon. Though coming close with slimes is quite dangerous, it can also be quite rewarding - slaying a slime nets a hefty amount of points. Unfortunately, while showing all adventurers at once, you can't see which slimes have been slain by which adventurers - just believe that if an adventurer passes through a slime without dying, they must already have killed it.
Finally, by generation 50, most adventurers have found out that the most advantageous route is to go east, slime-hunting. Though they are getting a lot further than the original iterations, it is not a gracious sight. Lots of sword-swinging. If I had let the animation go on, it would show them continously stumbling around the room, never realising that it might be time to leave once there are no more slimes.
Overall, this has been a very tedious and pretty disappointing project. I could spend a long time criticising the method, but I actually think the biggest problem is that I made the game too difficult.
Later on, I made it so that I could control one of the adventurers, and oh my god is it difficult. The tile-based movement, the unpredictable slimes and the unwieldy sword that can only hit in one of four directions. Again and again, I would stand diagonally to a slime in a terrible stale-mate. The second I left to be able to strike it, it would move too, winning the game by either escaping back to a new stale-mate or by moving into me, enveloping me and winning.
Some times, common sense might be just as important as the finer details of how I decided to train the neural network. Don't teach a computer to walk if you cannot crawl yourself.
It's possible that you didn't have enough nodes / layers in the network. What network topology did you use?
ReplyDeleteAlternatively, you might find different type of AI more successful. Have you considered Fuzzy Logic?
When you say that, I start to wonder. In my mind, the current AI is given such a simple task that a second layer wouldn't need to be necessary - if enemy to the right, swing sword to the right. But instead of being disappointed in the lacklustre results, I should probably, you know, actually have a neural network of at least two layers. Once I have time to train one, I'll try that out.
ReplyDeleteNot quite sure what you mean by fuzzy logic. As it is, the neural output is probabilistic - the most active end node is the most likely to be picked, though not always. This is done to avoid getting stuck walking into walls. Is that what you mean?