## My AI reads your mind and kicks your ass (part 2)

In the last post I discussed how it is possible to program a game Artificial Intelligence to exploit a player's unconscious biases using a simple mathematical model. In the karate game above, the AI uses that model in order to do the largest amount of damage. Give it a try! You get 10 points if you hit your opponent with a punch or a kick, 0 points if you miss, and 5 points if you block your opponent's move. As you play, the AI learns your strategy and adapts to knock you down as often as possible.

How does it work? According to decision theory, we need to maximize the *expected score*. To compute the expected score for an action 'x' (e.g., 'punch'), one needs to consider all possible player's moves, 'y', and weight the possible outcome with the probability of the player doing that move, i.e.

E[score for x] = sum_y P(y) * Score(y,x)

where P(y) is the probability of the player choosing action 'y' (obtained using last post's model), and Score(y,x) gives the score of responding 'x' to 'y'.

For example, in the karate game using a low kick has a priori the highest chance of success: you score in 3 out of 4 cases, and only lose 5 points if the opponent decides to block your kick. This is why, at the beginning, the AI tends to choose that move. However, if you know that the AI uses that move often, you will choose the kick-blocking move more often, increasing P(kick-block). This change will make the punch more likely to score points. As you play, the optimal strategy changes and the AI continues to adapt to your style.

With a bit of practice, you'll notice that you can compete with the AI and sometimes even gain the upper hand over it. This shows that you are in turn forming an internal model of the computer's strategy. I think that the game dynamics that results from this interaction makes the game quite interesting, even though it is extremely simple. Unfortunately, it's very rare to see learning AIs in real-life video games...

As always, you can download the code here.

**Update:** Instead of always making the best move, the AI now selects the move with a probability related to its score, which makes it less predictable. More details in the next post...

[ad#adpost]

Pragy AgarwalJuly 29th, 2013 - 14:39

Really nice article!

This approach is good for predicting the environment, by finding out patterns in input.

btw, I bet the game by randomising my moves.

That is not a weakness of the AI since the game simply becomes a random walk if either or both players randomize their moves, and given enough time, any outcome is possible with probability 1 in a random walk.

JevgenijDecember 17th, 2014 - 08:58

Hi, great article! Thanks!

Btw, can you guide me a bit. Let say we have ten actions {Nose (10 point), Jaw (8 points), Belly (6 points), Groin (4 points), Legs (3 points)} and blocks for all {Block Noise, B Belly, B Groin, B Legs}. All these 10 action grouped in Movement, like {Nose, Block Belly, Groin}, the order doesn’t matter. So if opponent performs such move {Legs, Groin, Block Nose} he will block player’s Nose action (because of Block Nose) and will score Legs and Groin (3 + 4). Player will score Nose and Groin (10 + 4). I tried you approach, but it doesn’t work well because actions can be grouped and also it’s too many actions, so the matrix is really huge. Could you suggest some solution how can I predict player moves?