CS 391 - Selected Topics: Game Artificial Intelligence
Readings |
Class | Month | Day | Topic | Readings (parenthesized reading are optional) |
1 | January | 19 | Course introduction, game definitions, combinatorial games, games of chance, strategic games, introduction to FreeCell | Course Information page, Wikipedia definitions of "game", Overview of Three Game Types |
2 | 21 | Solitaire games and classic AI search/planning, state, initial state, actions, state space, goal states, uninformed search algorithms (breadth-first search, depth-first search, depth-limited search, iterative-deepening depth-first search) | Russell & Norvig 3.1-3.4 | |
3 | 26 | (completion of uninformed search coverage) | ||
4 | 28 | FreeCell features discussion | ||
5 | February | 2 | Informed search and repeated state detection algorithms | |
6 | 4 | (in-class FreeCell solver design) | G. Heineman. Algorithm to Solve FreeCell Solitaire Games, (Elyasaf, Hauptman, and Sipper. Evolutionary Design of FreeCell Solvers) | |
7 | 9 | Minimax/Negamax, design discussion | Russell & Norvig 5.1-5.2, Wikipedia article on Minimax (Combinatorial Games section), Bruce Rosen's Minimax algorithm example, animation of Minimax algorithm | |
8 | 11 | Alpha-beta pruning, move ordering | Russell & Norvig 5.3, Bruce Rosen's worked example, Pieter Abbeel's YouTube demonstration, Move Ordering (skim top) | |
9 | 16 | (away at conference; in-class group project work) | ||
10 | 18 | Expectimax, introduction to Yahtzee, Upper Section Yahtzee, retrograde analysis | J. Glenn. An Optimal Strategy for Yahtzee | |
11 | 23 | Enumerating sorted rolls with the choose function, representation conversions (roll, rank counts, roll id) | Choose Function (Binomial Coefficient: Recursive Formula, Wikipedia) | |
12 | 25 | Yahtzee widgets, continued in-class implementation | ||
13 | March | 1 | Yahtzee reroll enumeration, transition probabaility computation | |
14 | 3 | Retrograde analysis revisited: within-widget computation | ||
15 | 15 | Game of Pig, optimal scoring play versus optimal winning play, value iteration | T. Neller, C. Presser. Optimal Play of the Dice Game Pig; see also Solving the Dice Game Pig (project PDF) | |
16 | 17 | Reinforcement learning, Markov decision processes, Bellman's optimality equations | Sutton, Barto. Reinforcement Learning: an introduction (chapter 3; Bellman equations in section 3.8) | |
17 | 22 | Piglet Solitaire exercise | ||
18 | 24 | Approach n game, n-armed bandit problem, exploration-exploitation tradeoff, epsilon-greedy selection, softmax selection | Sutton, Barto. Reinforcement Learning: an introduction (sections 2.1-2.3) | |
19 | 29 | UCB1 selection implementation, comparison of selection algorithms | Auer, Cesa-Bianchi, and Fischer. Finite-time Analysis of the Multiarmed Bandit Problem (introduction and Fig. 1) | |
20 | 31 | Imperfect information games, introduction to final project: Gin Rummy | ||
21 | April | 5 | Monte Carlo off/on-policy control (exploring starts/epsilon-soft), Monte Carlo Tree Search (MCTS), UCT | Sutton, Barto. Reinforcement Learning: an introduction (sections 5.3 and 5.4); C. Browne et al., A Survey of Monte Carlo Tree Search Methods, section 1 through 1.1, section 3. |
22 | 7 | Game Theory Basics, Nash Equilibrium, Pure and Mixed Strategies | GameTheory101.com "The Basics" videos #1-#7 | |
23 | 12 | Mixed Strategy Payoffs, Strong/Weak Dominance, Number of Equilibria | GameTheory101.com "The Basics" videos #8-#14 | |
24 | 14 | Correlated Equilibria | Wikipedia Correlated Equilibrium introduction and example, (optional: Kevin Leyton-Brown slides 26-34 on Correlated Equilibrium, GNU Linear Programming Kit Reference Manual appendices C (CPLEX LP Format) and D (glpsol usage)) | |
25 | 19 | Regret, Regret Matching, Extensive Form Games | An Introduction to Counterfactual Regret Minimization, Sections 1 - 2.3, Wikipedia extensive-form game article, (optional: Hart and Mas-Colell. A Simple Adaptive Procedure Leading to Correlated Equilibrium, paper sections 1-2, slides 21-70) | |
26 | 21 | Counterfactual Regret Minimization | An Introduction to Counterfactual Regret Minimization, Sections 3 - 3.3, slides, Wikipedia Kuhn Poker article | |
27 | 26 | In-class Course Evaluations, Neural Networks | slides | |
28 | 28 | Gin Rummy Player Presentations, Neural Networks |