CS 371 - Introduction to Artificial Intelligence Homework #5

Note: Work is to be done together in pairs.

Dynamic Programming and Value Iteration

0. Reading: Read the given handout Solving the Dice Game Pig: an introduction to dynamic programming and value iteration, and study the accompanying sample code.  This code can be downloaded from the Pig project webpage Description section.

1. Dynamic Programming Solving Pig Solitaire: Solve Pig Solitaire, the first exercise of section 2.4, according to this specification.  Answer the written exercises in your README file.

2. Value Iteration Solving Pig: Solve Pig, the first exercise of section 3.5, according to this specification.  Answer the written exercises in your README file.

3. Value Iteration Solving Two-Dice Pig: The rules of Two-Dice Pig are the same as the previous standard 1-die Pig except:

1. Two standard dice are rolled.
2. If neither shows a 1, their sum is added to the turn total.
3. If a single 1 is rolled, the player’s turn ends with the loss of the turn total.
4. If two 1s are rolled, the player’s turn ends with the loss of the turn total and score.

Solve Two-Dice Pig with value iteration according to this specification.   Answer the written exercises for the previous exercise applied to Two-Dice Pig in your README file.

Hints:

• Don't forget your README file. Most of the rubric points come from attention to the written questions.
• Don't forget base cases in your optimality equations. Otherwise, they're not grounded in terminal state values.
• Acyclic proofs aren't difficult here, but can't be hand-waved. The easiest approach is to (1) define a metric for progress (like an odometer meter reading), and (2) show that _every_ state transition results in strict progress of this metric.

Rubric:

Dynamic Programming - Pig Solitaire  /7
/1 (a) State space definition
/1 (b) Optimality equations
/1 (c) Acyclic state space proof
/2 (d) Computation of solution
/1 (e) Summarization/visualization and description of solution
/1 (f) Minimum number of turns to have >=50% chance of winning

Value Iteration - Pig  /6
/1 (a) State space definition
/1 (b) Optimality equations
/2 (c) Computation of solution
/2 (d) Summarization/visualization and description of solution

Value Iteration - Two-Dice Pig  /7
/1 (a) State space definition
/2 (b) Optimality equations
/2 (c) Computation of solution
/2 (d) Summarization/visualization and description of solution
--------------------------------
/20 Total

Todd Neller