Hard Light Productions Forums
Modding, Mission Design, and Coding => FS2 Open Coding - The Source Code Project (SCP) => Topic started by: jorgealdo on July 17, 2012, 12:26:14 am
-
How hard would be to adapt a neural network/evolutionary approach to game AI ?
How hard is to change fighter AI for one ?
Would this be possible ?
I think about this concept :
http://www.youtube.com/watch?v=C0jNMmdRMSo
http://www.youtube.com/watch?v=0Str0Rdkxxo&feature=related
Not a back-propagating neural network, but one where the connections and thresholds are set by genetic evolution, fitness would be measured by death/kill ratios.
Initally fighters would be pretty dumb, needing someone to take out them all during a course of months.
After AI gets to the point of being usefull, save the better genetic seed and deliver the game with that seed, all subsequent fighters would be genetic offsprint of that seed.
The neuron model would take a [x, y] vector of all enemy ships around as input (imagine one neuron per direction N, S, W, E, up, down etc) plus a hidden layer and the outputs would be turn left, turn right, run forward, fire etc... We can use a threshold model where a far distance is a low value (not passing the threshold) and a near distance is a high value (Passing the neuron threshold...)
What you think ?
-
Several years ago I had the idea to do just this sort of experiment. However, the job was going to be so complicated that I wanted a few additional coders to help out. The two coders that I recruited went AWOL and therefore the idea never went anywhere.
-
Several questions spring to mind:
1. How customizable would this AI be? (Right now, the AI behaviour can be tweaked very accurately)
2. How CPU-intensive would this AI be?
3. How would it affect the balance of existing missions?
4. Do you have any intention of supplying more to this endeavour than the initial idea?
5. Are there any other games around (No, Battlecruiser 3000 doesn't count) that use such an AI system?
6. How maintainable would the resulting code be?
-
5. Are there any other games around (No, Battlecruiser 3000 doesn't count) that use such an AI system?
Creatures I/II was supposed to be based on the concept of a learning AI
wasn't Black and White Learning AI or was that simulated? if it was learning AI this pdf might be of some interest (http://www.cs.rochester.edu/~brown/242/assts/termprojs/games.pdf)
-
3. How would it affect the balance of existing missions?
Irrelevant as long as the new AI doesn't replace the old one in an hardcoded manner. You just base your mod on the new AI if you decide to use it.
The rest of the concerns, though, are pretty valid, especially the "Do you have any intention of supplying more to this endeavour than the initial idea?" part.
-
Well, as I see it, maintaining two separate AI paths is about as mad an idea as maintaining two different graphics APIs.
if it was learning AI this pdf might be of some interest
That paper cites Daikatana as a positive example. Immediate disqualification.
-
if it was learning AI this pdf might be of some interest
That paper cites Daikatana as a positive example. Immediate disqualification.
fair enough
-
In any case, for most tasks you'd expect from a fighter, the current AI, especially with all the improvements known nowadays as "Fury's AI", is basically better than humans players. The player is still better for completing specific objectives, and you better not expect much from capital ship AI, but when it's about dogfighting... Well, I'd like to see you try to take on a Little Devil fighter in Insane, in dogfight range, with the same loadout as yours.
And for capship AI, there's nearly nothing you can't do with sexps, waypoints and scripts.
-
Don't get me wrong, there are areas where I personally believe the AI to be more stupid than it has to be (looking at you, bombers!), or where it could be more intelligent (sup, capships). I just don't think that going with a largely experimental tech with an unknown, but estimated to be long development cycle and unknown end results is the way to go here.
-
its really just fuzzy logic based ai.the general idea, from what ive read on the matter (i didnt read any of the articles posted here), is bascally you feed some analog signals (simulated with floats) into a network. each of these signals has a random entry point into a network of simulated neurons. some neural interaction goes on and another set of signals come out of the network. what the signals do or represent is not defined.
the network is a mass of virtual neurons. each neuron has a list of things its connected to, and potential value which starts at 0 and increases each time the neuron is 'zapped'. if the potential value passes some arbitrary threshold value, then all the neurons in the connection list get zapped. the amount of zap is the potential value of the neuron / number of entries in the connection list, and after zapping the potential value is set back to 0. you could also have a floating threshold that changes with time, to simulate a neuron going dumb. if a neuron is bored its threshold moves toward 0, and if its going off a lot the threshold moves away from zero. a neuron with a high threshold is healthy and can grow connections, and one with a low threshold is dumb and can only support a few. if a neuron is too active, say it overshoots its threshold by a large margin, can kill off some of its connections.
anyway you have a lot of these things in an array. they are all inited with random data within set ranges (potentials, thresholds, connections). each of the inputs is connected to a random neuron, zapping it with the input value. likewise each of the outputs has a random neuron bound to it, when the output is sampled it returns the potential value and sets it to zero. you simply iterate over the array, simulate each neuron in sequence, zap its list of connections if neccisary. essentially running in an endless loop. its probibly a good application for multithreading, as you want the brain to be thinking even in between applying stimuli and output sampling (you want it to use all available time on that core). it can go through thousands of simulation cycles a frame. when you want to apply stimuli, simulation is halted, the bound neurons are zapped, and the simulation resumes where it left off. sampling output follows the same rules.
thats the basis for how the network operates. the inputs and outputs can be anything. i would feed in raw facts. how much damage did i take last frame, how many times ive been hit this mission, current health status, number of hostile dots on its radar in relation to friendly dots. you could feed it advanced computer vision data, but the computer is fairly good at figuring out what the ai can see and what it cant. for outputs i would just use the data for behavioral characteristics about the ai. simulated fear or agression, bravery, desire to take risks, etc. the data is probibly not suitable for steering the ship or aiming. you would need a way more powerful brain for that. i really dont think it would do much other than make ai less predictable, and its also going to be a cpu hog, especially running it for multiple opponents. maybe some gpu power can be thrown at it.
-
A true Neural Network for fighter AI is perhaps overkill, since for the most part an AI's lifespan is within a single mission. The only real advantage of having an AI system that learns the player's combat style is when at least one of the hostile targets gets away, and therefore passing along vital combat data to their faction and allies so as to "adapt" to a new threat (the player).
But, as Nuke said, such a system would be a CPU hog. Even if the GPU could be pulled in to perform the spatial calculations, there would be a significant drop in framerate.
A simpler choice is to use a Fuzzy Inference system that takes in a finite amount of data, such as relative position of attackers, hull/shield status of self, others, etc. and then map it into fuzzy values by using a waveform function such as triangle or trapezoid. Then, a set of rules evaluates the fuzzy values, and outputs a weighted value for each rule.
Now, instead of traditional FLC's, the weighted values would NOT be combined into a single "defuzzified" value. Instead, the fuzzy value that has the highest magnitude dictates which action to take (which is associated for each rule).
However, again, such a system is computationally intensive, but can be averted somewhat by being only applied to particular actions, such as "Attack new target," "Do a barrel roll", etc., as well as be polled at a slower rate (like once per second) and/or when an action is complete.
Closing remark: Messing with the AI system is not easy, and any major change in the system needs to be thoroughly designed before any thought of code prototyping.
-
lua is probibly well suited for fuzzy logic and neural net simulation. it only knows floats and it's automatic memory allocation would come in handy, and it has mulithreading features built in. the downside is it has little or no integration with the games ai. you could give orders or do anything you could do with a sexp, but thats probibly about it. i might try it in my own game engine though and see how smart it is. but thats after physics, input, and bad guys.
-
Is it doable?
Yes. Sort of. You can make an AI that "learns" to some degree.
Is it worth it?
No, not by a long shot.
-
the most important aspect of neural nets that no one seems to have mentioned so far is that they need to be trained, it takes thousands and thousands of training sessions to get a neural net working at all, and while you could train them by having them figh each other over and over, getting them to the point that they can fight at all might prove challenging.
-
Actually, bob, that was mentioned right there in the first post. plz2readb4post, kthxbai
-
i figure the best approach to that is to pre-train a neural net. and then once you have it acting right, dump the entire neuron array to file. use this file and others as ai templates, give them a name that fits their personality, such as dumbass and retard. you can load em and have em start thinking right where they left off.
-
But, and this is the important question none of you have answered yet, how would this be any better than the current AI? How much work would you have to put into training the AI before you have a set of seeds that can cover the same amount of configurability that the current AI has?
In the end, the difference between a neural-net-driven AI and a normal procedural one is something the end user is probably never ever going to notice. So the other important question is, why bother?
-
i have answered more or less. it wouldnt. it would be an interesting tech demo at best. the fact that its going to hog up an entire cpu core kinda makes it not worth it. definately need some kind of neural net on a chip that can run at a much higher clock than a cpu, it doent even matter if the device is prone to error, it would probibly improve performance. you could probibly have a rudimentary system based on an fpga perhaps. intresting expirement: yes, something i want in freespace: no
-
But, and this is the important question none of you have answered yet, how would this be any better than the current AI? How much work would you have to put into training the AI before you have a set of seeds that can cover the same amount of configurability that the current AI has?
In the end, the difference between a neural-net-driven AI and a normal procedural one is something the end user is probably never ever going to notice. So the other important question is, why bother?
I was just about to ask the same question. Since the result of such a neural net would basically translate to different settings of the already existing AI parameters. What you would get (if you get the neural net working and learning) would be similar to just sliding the difficulty setting and AI classes up during the mission. It would not result in fundamentally new AI maneuvers.
EDIT: I get the idea that a self-learning AI would be cool, but for a neural net to work in that way, you'd have to first parametrize a LOT more of the AI behavior, down to the nuts'n'bolts desired pitch/yaw/bank/speed values. And that would increase the complexity by a few orders of magnitude
-
Actually, bob, that was mentioned right there in the first post. plz2readb4post, kthxbai
I don't think the 'thousands and thousands' part was.
-
Bob, it was implied here:
Initally fighters would be pretty dumb, needing someone to take out them all during a course of months.
-
To be fair, that sentence is pretty hard to read.
-
To be fair, that sentence is pretty hard to read.
Yep.
Oh, and Nuke, the OP suggested using a "seed," which probably meant something like the files you suggested (that or some generic value...)
-
But, and this is the important question none of you have answered yet, how would this be any better than the current AI? How much work would you have to put into training the AI before you have a set of seeds that can cover the same amount of configurability that the current AI has?
In the end, the difference between a neural-net-driven AI and a normal procedural one is something the end user is probably never ever going to notice. So the other important question is, why bother?
My proposal was to create a plugin for an entirely alternate AI system, one built using genetic coding (a form of genetic algorithm evolution). Theoretically you could accomplish this by mapping function building blocks (either in Lua or C++) to "genes", then mutating them in certain ways to evolve a certain AI behavior (attack, evade, etc.)
You would need a "fitness function" to determine the suitability of each "chromosome", but this could be tied to such things as hull strength and length of time before getting destroyed. You would also need to add a hook into FreeSpace to run a mission with this AI over and over and over... for long periods of time. Possibly weeks. Possibly even months. However, if this had been started when I proposed it, two years ago, we could have had something by now. :nervous:
Upon reflection, this may actually be somewhat different from what the original poster was asking about.