What I really, really want to work with is AI, and more specifically, competitive AI. All the stuff to do with PokeHearth really comes down to see if I can make an AI with surprising strategic prowess. PokeHearth was a partial success, but only when it comes to the card choices, not so much with the AI itself.
Why did the PokeHearth AI fail? I used to try evolving forth an AI, but it clearly became obvious that it had some big problems to overcome:
Starcraft and Macro Strategy
Inspired by Starcraft, a less random game, I tried to make an extremely simplified game-model. Overall, the AI would have to balance four decisions:
Why did the PokeHearth AI fail? I used to try evolving forth an AI, but it clearly became obvious that it had some big problems to overcome:
- The AI had to be general-purpose, since there were too many combinations of cards to hold a "solution" to every situation.
- Instead, the AI would simply work from a set of weights and try to find the most optimal solution via a prefigured, non-AI algorithm. This limits the uniqueness of AI.
- Determining if one AI is better than another is very difficult, since, depending on luck, the worse AI might have a higher winrate than another.
- This can be circumvented by having the AI play more games before comparisons are made.
- What AI is best will not only depend on the cards being dealt, but on the decks being played against. How does one represent the knowledge of "this deck is control, I must play aggressively"?
- This can be solved by either having either 18 different AIs for each deck, or a more complicated neural system.
- The AI plays against other intelligent AI which, while evolving, will change what solutions are optimal.
- This is what makes it interesting and impossible all at once. If this were the only problem, perhaps a solution could be found, but with the above problems too...
This weekend, I had an idea for another AI. Working on it, I saw that while I circumvented some of these problems, others kept rearing their head.
Starcraft and Macro Strategy
Inspired by Starcraft, a less random game, I tried to make an extremely simplified game-model. Overall, the AI would have to balance four decisions:
- Using resources to create worker units to increase ressource generation.
- Using resources to create combat units for defence and offence.
- Using resources to build structures for making combat units.
- Attacking with delayed reinforcements or collecting a defensive army.
Of course, the neural net and the game rules are a lot more complicated than this. Still, the central idea is this simple balancing between building infrastructure or combat units, and then letting a neural network evolve to solve the problem the best way possible.
Okay, so the project is far from graphically minded, but I hope you get the idea. But this is before we even start evolving the network. What might happen if we let it evolve for several generations?
Reductio Ad Absurdum
is a faux-Latin phrase describes when your model becomes so reductionary, the result is absurd. Reduction is a necessary part of models, whether we are talking about scientific models or neural networks. A model must necessarily be compacted compared to the original - for instance, a map of a country at a 1:1 scale would be as large as the country and not useful at all. For a more useful map, you reduce not only the size, but also what is and is not rendered - a circle for a city instead of a thousand dots of pebbled roads.
Reductio ad absurdum takes place in, er, models of more complex systems than a city map. For instance, one can consider killing a mouse and suspending one of its muscles in an electrified vat a very simplified model of human exercise. Nonetheless, it might be necessary if you want to study molecular changes due to exercise. You cannot do that in a live human. But the question arises if the results you find in the reduced model even apply to the full system.
Neural networks both require and perform reduction. In the classic number-recognition example, where a neural net is trained to distinguish the numbers 0-9 written in human handwriting, the task of the neural net is to reduce the massive amount of colour information into just ten pieces of information: which of the numbers it looks most like.
But this is just the reduction performed by the neural network. On top of this, there also is a more ontological reduction inherent in the setup of the neural network. The goal of the neural net is simple: To produce a singular result. The input has been simplified as well. This neural network might become very sophisticated at describing handwritten numbers. But it can never do anything beyond that. It cannot teach us anything about the world apart from how to recognise handwritten numbers that have been turned into a digital image of the correct format, size and colour. Just like analysis of a rodent muscle fiber struggles to teach us anything aboutt he world apart from what it does itself.
This is not an attack
I do not mean to criticise neural networks with this. I mean, rather, to point out the necessary attention to the functioning of reduction as it pertains to working with neural networks.
When I first started out working with neural networks, I believed I could simply set it up without any thought and then watch as wonderful results came back out. But neural nets are not general purpose. They cannot explain complicated systems. They are most useful in situations where both inputs and outputs are simplified. Neural networks require reduction.
"Reductio ad nauseam" is not a critique that I direct at neural networks, though. There are simple systems that remain meaningful even after reduction has been applied. Who cares if my network for recognising handwritten numbers has a soul? I don't need it to have one.
This is instead a case where I am guilty of reductio ad nauseam.
Simple problems, simple answers
The first time I worked with neural networks, I left my computer to run as I went to work, and I came back from work with bated breath. What had it produced?
This project did not offer much when it comes to design. However, the interesting result, after 118,320 duels, is the red circle beneath HP. This tells the Warlock hero that its own health is not very important. I was so happy when I saw this! Because it was what I had predicted all along, the proof that it worked. And then the question came up - if I had known this would happen, was it interesting at all? Could I not just have let the neural network know this from the start?
So I made the Starcraft model as described above. After a dozen of generations, almost all AI started out building workers, then switching to attacking units at around 10 workers. Then, after another couple dozen generations, they now went up to 14 workers.
Again, proof that it worked! But it was not a very interesting result. It is, after all, obvious that if you have just slightly more economy than the opponent, you will win. If I had implemented some chance of a strong mutation, at some point it might jump down to an all-out aggressive strategy to counter the greedy economy builds.
I had created a very simple model of a game, and therefore, I ended up with a neural net that found a very simple solution.
Of course, it is a very simplified model, and there is still much more time to improve it, make it more complex. But as I do so, the neural network will be weakened. First, because the time to run a single game will rise as the program becomes heavier. But second because there will be so many bad strategies to distract the neural network from finding interesting, workable solutions, that are not just simple counters to the main strategy.
The true goal is to create a neural network that not only reacts to itself, but also to the opponent. But there is no certainty that this will ever happen. It might be too complex for the AI to solve, unless I find a clever way to reduce the complexity further.
In other news, maybe I will post more fully about the RTS AI if I have enough time to make it work despite the added complexity.
When I first started out working with neural networks, I believed I could simply set it up without any thought and then watch as wonderful results came back out. But neural nets are not general purpose. They cannot explain complicated systems. They are most useful in situations where both inputs and outputs are simplified. Neural networks require reduction.
"Reductio ad nauseam" is not a critique that I direct at neural networks, though. There are simple systems that remain meaningful even after reduction has been applied. Who cares if my network for recognising handwritten numbers has a soul? I don't need it to have one.
This is instead a case where I am guilty of reductio ad nauseam.
Simple problems, simple answers
The first time I worked with neural networks, I left my computer to run as I went to work, and I came back from work with bated breath. What had it produced?
So I made the Starcraft model as described above. After a dozen of generations, almost all AI started out building workers, then switching to attacking units at around 10 workers. Then, after another couple dozen generations, they now went up to 14 workers.
Again, proof that it worked! But it was not a very interesting result. It is, after all, obvious that if you have just slightly more economy than the opponent, you will win. If I had implemented some chance of a strong mutation, at some point it might jump down to an all-out aggressive strategy to counter the greedy economy builds.
I had created a very simple model of a game, and therefore, I ended up with a neural net that found a very simple solution.
Of course, it is a very simplified model, and there is still much more time to improve it, make it more complex. But as I do so, the neural network will be weakened. First, because the time to run a single game will rise as the program becomes heavier. But second because there will be so many bad strategies to distract the neural network from finding interesting, workable solutions, that are not just simple counters to the main strategy.
The true goal is to create a neural network that not only reacts to itself, but also to the opponent. But there is no certainty that this will ever happen. It might be too complex for the AI to solve, unless I find a clever way to reduce the complexity further.
In other news, maybe I will post more fully about the RTS AI if I have enough time to make it work despite the added complexity.
Comments
Post a Comment