CS 391 - Selected Topics: Game Artificial Intelligence
Readings

Note: This syllabus is tentative and subject to change.  Each reading assignment should be completed before the class on the date indicated.  If a reading assigned in class does not match the reading assignment here, the reading assigned in class supersedes. 

 
Class Month Day Topic Readings (parenthesized
reading are optional)
1 January 19 Course introduction, game definitions, combinatorial games, games of chance, strategic games, introduction to FreeCell Course Information page, Wikipedia definitions of "game", Overview of Three Game Types
2 21 Solitaire games and classic AI search/planning, state, initial state,  actions, state space, goal states, uninformed search algorithms (breadth-first search, depth-first search, depth-limited search, iterative-deepening depth-first search) Russell & Norvig 3.1-3.4
3   26 (completion of uninformed search coverage)  
4   28 FreeCell features discussion  
5 February 2 Informed search and repeated state detection algorithms  
6   4 (in-class FreeCell solver design) G. Heineman. Algorithm to Solve FreeCell Solitaire Games, (Elyasaf, Hauptman, and Sipper. Evolutionary Design of FreeCell Solvers)
7   9 Minimax/Negamax, design discussion Russell & Norvig 5.1-5.2, Wikipedia article on Minimax (Combinatorial Games section), Bruce Rosen's Minimax algorithm example, animation of Minimax algorithm
8   11 Alpha-beta pruning, move ordering Russell & Norvig 5.3, Bruce Rosen's worked example, Pieter Abbeel's YouTube demonstration, Move Ordering (skim top)
9   16 (away at conference; in-class group project work)  
10   18 Expectimax, introduction to Yahtzee, Upper Section Yahtzee, retrograde analysis J. Glenn. An Optimal Strategy for Yahtzee
11   23 Enumerating sorted rolls with the choose function, representation conversions (roll, rank counts, roll id) Choose Function (Binomial Coefficient: Recursive Formula, Wikipedia)
12   25 Yahtzee widgets, continued in-class implementation  
13 March 1 Yahtzee reroll enumeration, transition probabaility computation  
14   3 Retrograde analysis revisited: within-widget computation  
15   15 Game of Pig, optimal scoring play versus optimal winning play, value iteration T. Neller, C. Presser. Optimal Play of the Dice Game Pig; see also Solving the Dice Game Pig (project PDF)
16 17 Reinforcement learning, Markov decision processes, Bellman's optimality equations Sutton, Barto. Reinforcement Learning: an introduction (chapter 3; Bellman equations in section 3.8)
17 22 Piglet Solitaire exercise
18   24 Approach n game, n-armed bandit problem, exploration-exploitation tradeoff, epsilon-greedy selection, softmax selection Sutton, Barto. Reinforcement Learning: an introduction (sections 2.1-2.3)
19   29 UCB1 selection implementation, comparison of selection algorithms Auer, Cesa-Bianchi, and Fischer. Finite-time Analysis of the Multiarmed Bandit Problem (introduction and Fig. 1)
20   31 Imperfect information games, introduction to final project: Gin Rummy  
21 April 5 Monte Carlo off/on-policy control (exploring starts/epsilon-soft), Monte Carlo Tree Search (MCTS), UCT Sutton, Barto. Reinforcement Learning: an introduction (sections 5.3 and 5.4); C. Browne et al., A Survey of Monte Carlo Tree Search Methods, section 1 through 1.1, section 3.
22   7 Game Theory Basics, Nash Equilibrium, Pure and Mixed Strategies GameTheory101.com "The Basics" videos #1-#7
23   12 Mixed Strategy Payoffs, Strong/Weak Dominance, Number of Equilibria GameTheory101.com "The Basics" videos #8-#14
24   14 Correlated Equilibria Wikipedia Correlated Equilibrium introduction and example, (optional: Kevin Leyton-Brown slides 26-34 on Correlated Equilibrium, GNU Linear Programming Kit Reference Manual appendices C (CPLEX LP Format) and D (glpsol usage))
25   19 Regret, Regret Matching, Extensive Form Games An Introduction to Counterfactual Regret Minimization, Sections 1 - 2.3, Wikipedia extensive-form game article, (optional: Hart and Mas-Colell. A Simple Adaptive Procedure Leading to Correlated Equilibrium, paper sections 1-2, slides 21-70)
26   21 Counterfactual Regret Minimization An Introduction to Counterfactual Regret Minimization, Sections 3 - 3.3, slides, Wikipedia Kuhn Poker article
27   26 In-class Course Evaluations, Neural Networks slides
28   28 Gin Rummy Player Presentations, Neural Networks  
  
Todd Neller