Subscribe to download
AI Trainer

A template for Machine Learning. This example lets you train a Neural Network using a Genetic Algorithm.


You have no idea what those words mean? Don't worry you can still try to stick an AI on various objects and attempt to train them to perform basic tasks with this mod :p You will have to be patient though, it can take a lot of time for it to learn any interesting behavior...

TOOLS
- AI Trainer: You can attach AIs to objects and train them to perform a given task.

KEYBOARD CONTROLS
- [LeftClick]: Attach/Detach AI to object.
- [MiddleClick]: Copy AI from object.
- [X]: Activate/Deactivate AI on object.
- [C]: Apply current AI and objective to all existing AIs.
- [R]: Start/Stop training.
- [E]: Pause/Resume training.
- [K]: Remove all AIs.
- [O]: Options.

CHARACTERISTICS
- You can only attach an AI to a free floating object.
- There are tree objective you can train your AI to do:
-- Balancing itself on a corner
-- Following the player
-- Avoiding nearby objects
- Pick the objective you want in the options, only one objective can be learnt at a time.
- The AIs only control objects by applying rotation speed to them, so if the object you try to control is already under the effect of some other script that prevent it from rotating, then the AIs from this mod won't be able to do anything to it.
- The "AI" used in this mod is a neural network that you can train to perform various tasks (only one task at a time though).
- The training method used is a Genetic Algorithm, which reproduces the rules of natural selection to find the best specimens in a group.
- There is another optimization method called gradient descent that you can enable on top of the genetic algorithm to try and get good results faster (enabled by default).
- Each neural network structure is suited for a given objective, so you need to ensure they match in the options when you change objective (an error is displayed if they don't match).
- A starting population of 20 should be good (or down to 10 if you get low FPS).
- With the default settings it can take up to 400 generations to start getting good results.
- You may want to make some free space around the learning objects and make sure they are not stuck (objects in water will train badly except for balance objective).
- You may want to train the objects on flat ground to avoid learning weird behavor due to the terrain slopes or being stopped by walls (except for avoid objective).

KNOWN BUGS
- Some characters won't appear in the text field when I type them: This is a limitation of the current modding API.
- Text in text fields cannot be copy-pasted: This is a limitation of the current modding API.
- AIs may fail to learn the objective given to them even after a long training session.
- An AI was doing really well at the objective and then started to be bad: You may have been lucky to see it behave correctly in a very particular situation, it takes a long time for it to learn to behave correctly in all situations.
- An AI works really good on one type of objects but bad on others: You may want to include more diverse type of object in the population used for training the next time your train it.

DEV NOTES

Originally posted by About this mod:

Well, I thought my previous mod was the mod I spent the most time developing... Look like this one wins the cake and beats it largely XD
I started this mod both because I thought it would be a funny thing to have in the game and also to actually learn about this topic myself: I heard a lot about neural networks, but never really tried to use them before, so this was also a good opportunity to learn a little how they work :)
Turns out building a neural network is not that hard, yet I did spend some time on the UI for this part. I wanted the players to be able to fully edit the neural network structure and to be able to share them with each other. I partially succeeded to make the network editable through a brand new (and ugly) text input field, sadly there is a lot of limitations with it that make it hard to achieve the full edition objective: First of all the game does not recognize all special characters your keyboard can print, which is not helping when you try to edit a json structure... And then I couldn't get it to display only the text that is inside the text field, resulting in the ugly leak of text outside the field if you enter a too long name >_< But at least you can scroll in it with the arrows, so you can at least see the full text. About the sharing part sadly I soon realized that there was no way to copy/paste text or save/load text to a file with the modding API, so I had to abandon this part for now.
Getting the neural network to produce some outputs based on the inputs you give it is quite simple too, this is what is called "forward propagation" and I was able to get this working quite fast.
The tricky part came when I tried to train it (to make it better at achieving a given taks). The first think I did was to implement a genetic algorithm, which is not easy but as I had done it before it was quite simple for me to take my old code and adapt it to the game :p Then I tested it to see if the AI seemed to learn anything over generations, but the results were decieving: I could not notice much improvement over time, even on a large amount of generations (around 100). After thinking this through, I understood that getting the actual good weights and bias in the whole neural network using genetic algorithm would simply take forever as the algorithm only keeps using the weights and bias that were available at the start of the training session, it simply shuffles them across all networks in the populaton.
So I decided to add another optimization algorithm on top of it to try and make them learn faster. The algorithm I selected is called Deep Q-Learning and lets you train a network to select one action to execute along a predefined list of actions based on the inputs and a reward value you give it to say how good the AI is doing at the task. So I changed a little the networks structure I had originally to accomodate this new setup and started implementing the algorithm itself. This took me forever as most examples online are using libraries that does all of the annoying work for you (like computing gradients of fonctions and applying them in the right order to get the new weights values), sadly I had no such lib un Lua so I had to implement all of those on my own... I'm still unsure if my version is fully correct and fully working as intended, but at least now I have something that is no longer raising errors every loop X) And I managed to train at least one network to do the task it was intended to do in a not so bad way (the balancing one), so that's good enough for me XD