CS 371 - Introduction to Artificial Intelligence Pre-Class and In-Class Activities

To start, you will need to be working on a machine with Eclipse and a web browser (or will need to hazard remotely working on our lab machines). Instructions for setting up your machine for remote work are at the page Tools for Working Remotely. If you would have difficulty accessing/configuring such a machine please contact me.

• CS 371 home page
• Tools for Working Remotely
• My Office Hours are listed on the course information page.
• Remember to do all readings and any assigned video exercises before each class.
• Colloquia attendance will still be required. Make a habit of checking your gettysburg.edu email at least daily to get talk announcements.
• Homework submission:
  1. Make sure you have the following in the same, single directory:
     • README file containing: ​the Honor Pledge, your names, your student ID numbers​, and any answers to required written portions of an assignment
     • any required .java source files. (There should be no “package” line at the top – use the “default package” by leaving the package blank in the New… Class form.)
  2. Change to that directory in a BASH terminal window.
  3. Enter the command “submit371 hw1” (or "hwN" for the Nth assignment)
  4. Some seconds after doing so, you should see a file named “feedback.txt” appear in that same directory, providing confirmation of your submission and feedback concerning the submission. You can read this using command “less feedback.txt”. When using the less command, you can go forward one line pressing Enter, forward one page pressing Space, back one page pressing “b”, and quit from less pressing “q”.
• Before class 1 ():
  • Think about how you would define "intelligence" and be prepared to discuss your definition.
  • Assigned skimming and readings
• Class 1 ():
  • Download and import this Eclipse project for class exercise: cs371-uninformed-search.zip
• Before class 2 ():
  • Review of class 1 uninformed search reading topics in more detail: (1:04:41)
    • The Uninformed Search Problem (15:47)
    • The Buckets Problem (SearchNode.java, BucketsNode.java) (16:19)
    • The Triangular Peg Solitaire Problem (PegSolitaireNode.java) (9:09)
    • Uninformed Search Algorithms (6:07)
    • Implementing Breadth-First Search (Searcher.java, BreadthFirstSearcher.java) and Computational Search Criteria (12:25)
    • Testing Search (SearchTest.java) (4:54)
  • Assigned readings
  • Watch and program along to these videos with new material: (36:24)
    • Implementing Depth-First search (DepthFirstSearcher.java) and Computational Search Criteria Tradeoffs (14:15)
    • Depth-Limited Search Specification (DepthLimitedSearcher.java) (3:14)
    • Iterative-Deepening Depth-First Search Specification (IterativeDeepeningDepthFirstSearcher.java) (3:27)
    • Computational Search Criteria Tradeoffs (continued) (15:28)
• Class 2 ():
  • Questions and Answers for readings and videos
  • Download "Suit Yourself Solo" problem specification and generator: SuitYourselfSoloGenerator.java
  • Implement SearchNode subclass SuitYourselfSoloNode.java for problem "Suit Yourself Solo" with a constructor having the same parameters as the problem generator. Optimization hint: Don't keep copying and updating the card deals. Instead, let your changing state be an array with the number of cards remaining in each pile.
  • Find the optimal (shortest) solution for "new SuitYourselfSoloNode(4, 13, 5, 0L);"
• Before class 3 ():
  • Watch and program along with these videos:
    • A different approach to the buckets problem (7:59)
    • What causes the inefficiency in our search (4:08)
    • A spectrum of design choices to address this (11:14)
    • Implementing TilePuzzleNode.java (40:39)
  • Complete IterativeDeepeningDepthFirstSearcher.java for use in Class 3.
• Class 3 ():
  • HW1 due class 4 - Any questions?
  • Create test code that runs IterativeDeepeningDepthFirstSearcher multiple times with randomly generated TilePuzzleNodes with size 4 and a number of shuffles that makes search take a long time (but with success).
  • See the performance impact of the simplest form of repeated state detection that we covered in the videos.
  • Try different forms of repeated state detection as time permits. We'll share our outcomes at the end of class.
• Before class 4 ():
  • Assigned readings
  • Watch and program along with these videos:
    • Motivation for informed search (6:40)
    • Functions, f, g, and h (7:23)
    • Best-First Searches: Greedy, Uniform, and A* Search Algorithms (16:01)
    • Java project classes (18:02) and BestFirstSearcher.java correction (2:48) from cs371-informed-search.zip
    • HTilePuzzleNode.java uninformed implementation (15:34) - We will discuss and implement admissible heuristics in class.
• Class 4 ():
  • Overview HW2
  • Discuss potential admissible heuristics for HTilePuzzleNode
  • Implement different admissible HTilePuzzleNode heuristics and compare performance
• Before Class 5 ():
  • Assigned readings
  • Complete the Stochastic Local Search (SLS) Video Tutorial with associated implementation before class
• Class 5 ():
  • Discuss representation of Preferred Group Formation problem.
  • Implement challenge problem in groups: Preferred Group Formation (group-input.txt)
  • Discuss tuning of Simulated Annealing for Preferred Group Formation problem.
• Before class 6 ():
  • Complete Preferred Group Formation problem implementation and optimize given sample problem (group-input.txt). Be able to share your share your solution in class.
• Class 6 ():
  • Check solutions for (group-input.txt).
  • SLS Q&A
  • Overview HW3
  • Open ended SLS problem: Suppose a dozen or more students want to have pizza ordered for their event and you want to place the best pizza order for the group. Assume that pizza toppings can be varied by half-pizza. Now students have many different preferences for (or against) toppings. Students also have different appetites. Design a system to get student preferences and place and order that will in some sense be best for the group.
• Before Class 7 ():
  • Assigned readings
  • Watch the following FYS 187-4 videos on game-tree reasoning:
    • game trees (21:02)
    • alpha-beta pruning (25:58)
• Class 7 ():
  • Alpha-beta pruning exercises in groups using common exercise tree. Optional: You can generate your own practice exercises with GenAlphaBetaProblem.java.
    • Group 1: A=3 B=2 C=4 D=2 E=4 F=2 G=4 H=3 I=4
    • Group 2: A=4 B=2 C=4 D=2 E=1 F=3 G=2 H=1 I=5
    • Group 3: A=4 B=5 C=5 D=3 E=5 F=2 G=1 H=4 I=1
    • Group 4: A=3 B=5 C=3 D=3 E=1 F=2 G=2 H=1 I=1
  • Overview HW4 and introduce code base.
  • Play Mancala and discuss strategy.
• Before Class 8 ():
  • Assigned readings
  • Watch the following FYS 187-4 videos on heuristic evaluation of game-tree nodes:
    • Note: The FYS 187-4 exercises I describe are not assigned for CS 371, but would make good practice.
    • heuristic evaluation (4:27)
    • heuristic features (11:25)
    • Optional if covered in Class 7:
      • Video presentation of the rules of Mancala (a.k.a. Kalah) (5:52)
      • Video demonstration game of Mancala (6:20)
• Class 8 ():
  • Heuristic feature discussion: Connect 4
  • In groups: Mancala play and heuristics brainstorming
    • Download Ludii game system (platform independent Java .jar file) or get Ludii-1.3.1.jar locally.
    • Start the game program with command "java -jar Ludii-1.3.1.jar" (or whichever more recent version exists), then File → Load Game (or Ctrl-L) → Games → board → sow → two rows → FairKalah.
    • Play games and discuss possible features as you play.
• Before class 9 ():
  • Assigned readings
• Class 9 ():
  • In-class discussion of assigned reading on time management.
  • Insights from Todd W. Neller, Taylor C. Neller FairKalah: Towards Fair Mancala Play, Computers and Games: International Conference, CG 2022, Virtual Event, November 22-24, 2022, Revised Selected Papers, Springer Cham, 2022. https://doi.org/10.1007/978-3-031-34017-8
  • If time, guided Mancala project work in assigned pairs.
• Before class 10 ():
  • Assigned readings
  • FYS 187-4 video (not optional while remote): chance node, expectimax, and optimal Pig play reasoning
  • Read Poker Squares play sheet. I'll bring sheets to class for us to play.
• Class 10 ():
  • Worked expectimax examples
  • Discussion of utility, and the importance of utility definition.
    • "The difference between utopic and dystopic visions of AI in sci-fi is in the definition of the utility function."
    • "To exclude non-quantitative values from utility functions is to assign them a utility of 0."
  • Parameterized Poker Squares play with Poker Squares play sheets
• Before class 11 ():
  • Assigned readings
• Class 11 ():
  • Parameterized Poker Squares abstraction
  • Use of abstraction: abstract problem, solve abstracted problem, apply abstract solution to original problem
  • Expectimax variants:
    • Depth-limited
    • Iterative-deepening
    • Chance sampling
      • Breadth-limited
      • Iterative-widening
  • Connect Four play demonstration - install Eclipse project cs371-bandit-connect4.zip
    • Run Connect4.java with settings: 6 Rows, 7 Columns, Black AI Player (unchecked), Red AI Player (checked), UCT Player (unchecked, then checked for next game later in class), then Play
  • n-Armed Bandit demonstration activity - Run NArmedBanditGame.java:
    • Set random seed (y/n)? y
    • Seed? 1234
    • (1) Normal mean distribution, (2) Logorithmic mean distribution, or (3) Given mean distribution? 1
    • Number of arms? 10
    • Total number of pulls? 100
  • Overview of exploration/exploitation dilemma, n-armed bandit problem, action selection algorithms, and Monte Carlo Tree Search
• Before class 12 ():
  • Assigned readings
• Class 12 ():
  • Partial presentation on the analysis of the dice game Pig (slides)
  • Summarization of dynamic programming (DP) pattern using project example code examples.
  • Overview of HW5 DP exercise.
• Before class 13 ():
  • Assigned readings
• Class 13 ():
  • Approach n spreadsheet example of dynamic programming: approach-10-with-6-sided-die-blank.ods or Google Sheet version
  • Overview of Value Iteration
  • Expected Chutes and Ladders turns computed with value iteration: ChutesAndLaddersSolverInClass.java
• Before class 14 ():
  • Bellman's Optimality Equations and Value Iteration
  • Overview of PigletSolver.java
  • Overview of homework value iteration exercises
• Class 14 ():
  • Complete Chutes and Ladders Solver.
  • Download and complete class exercise HogSolverInClass.java
  • Midterm exam expectations overview
• Before class 15 ():
  • Study for midterm exam to be given in class 15.
• Class 15 ():
  • Moodle midterm in class
  • Midterm programming questions due before class 17.
• Before class 16 ():
  • Assigned readings
  • Start midterm programming exercises and bring any questions about them that you have to class.
• Class 16 ():
  • Review of Bellman's Optimality Equations for Markov Decision Processes (MDPs) from Reinforcement Learning: An Introduction, 2nd ed. by R. Sutton and A. Barto
  • V(s) and Q(s,a) backup diagrams
  • Q-Learning algorithm
  • Demonstration of Q-Learning implementation for Piglet Solitaire
• Before class 17 ():
  • Assigned readings
  • Finish and submit midterm programming exercise code.
• Class 17 ():
  • Q-Learning
• Before class 18 ():
  • Assigned readings/activities
• Class 18 ():
  • Q-Learning implementation for HW6
• Before class 19 ():
  • 4th Hour Robotics
• Class 19 ():
  • Machine Learning (ML) CS Core - Session 1:
    • Machine Learning work largely falls into three categories: Supervised Learning, Unsupervised Learning, and Reinforcement Learning.
    • In Supervised Learning, we are given input-output pairs and we learn models to predict outputs from inputs.
    • Supervised Learning to predict discrete class outputs is called Classification. Simple techniques include k-Nearest Neighbors and Decision Tree classification.
    • Supervised Learning to predict continuous value outputs is called Regression. One simple technique is Linear Regression.
    • Unsupervised Learning is given unlabeled data and seeks to uncover hidden patterns, structures, or features within the input data itself. Clustering is an example task of Unsupervised Learning, and k-Means Clustering is a simple clustering algorithm.
    • Reinforcement Learning seeks to learn a mapping from states to actions (i.e., a policy) so as to maximize expected rewards through a stochastic sequence of state-action transitions. The Exploration-Exploitation Dilemma is a fundamental tension in action selection during reinforcement learning, and different techniques such as ε-Greedy and Sequential Halving seek to address this tension. Q-Learning is an algorithm that enables an agent to learn through repeated experiential episodes.
• Before class 20 ():
  • 4th Hour Robotics
• Class 20 ():
  • Machine Learning (ML) CS Core - Session 2:
    • Underfitting occurs when a machine learning model is too simple to capture the underlying patterns in the data, resulting in poor performance on both the training and testing datasets.
    • Overfitting occurs when a machine learning model learns the training data too well, capturing noise and outliers instead of the underlying patterns, which results in excellent performance on the training set but poor generalization to unseen data.
    • The Bias-Variance Tradeoff is the conflict between minimizing two sources of predictive model error:
      • bias, error from the inadequacy of the predictive function that is chosen for the predictive model, and
      • variance, error introduced by the model's sensitivity to small fluctuations in the training data.
    • Bias and variance are often traded off by:
      • Observing which model complexity settings best predict held-out testing data
      • Penalizing model complexity via the loss/performance metric
    • Common performance measures:
      • Regression: mean absolute error, mean squared error, root mean squared error, R squared (coefficient of determination; linear regression only)
      • Classification: accuracy, precision, recall, F1 score
    • With significant tuning of model hyperparameters, partition the data into train/validation/test sets. Iteratively train different hyperparameter models with the train set while measuring generalizable performance with the validation set. Finally, test the best hyperparameter model with the test set.
    • It's important to understand what one's model has learned in order to guard against biased ML models.
• Before class 21 ():
  • 4th Hour Robotics work
  • Review prior class material
• Class 21 ():
  • Machine Learning (ML) CS Core - Session 3: Part 1 of 2
    • Data Preprocessing is important prerequisite work that extends beyond the goals of data wrangling (providing accurate, consistent, anomoly-free data) to additional steps that tailor the data to ML models:
      • Handling Missing Values: Gaining understanding of the distribution of missing values and then either imputing (predicting) them or flagging them as missing
      • Encoding Values: Assigning categorical values to one-hot encoded features, and assigning continuous values to discrete bins
      • Normalizing/Standardizing Values: Transforming data values to same-magnitude ranges or normal distributions so as to avoid feature importance bias from features having different scales/distributions
      • Feature engineering: Deriving new input features that can aid the ML model computation
    • No-Free-Lunch Theorem: Without assumptions of the apriori distribution of target functions to be approximated, no machine learning algorithm can be claimed to be superior to another.
    • ML is theoretically undecidable.
    • Common sources of ML error include noise, bias, and variance.
    • There are fundamental tradeoffs between the expressiveness (complexity) of a ML algorithm's hypothesis space (i.e. functions that can be learned), the amount of training data needed, and the expected performance of the ML algorithm.
  • Q-Learning Project Demonstrations
• Before class 22 ():
  • 4th Hour Robotics work
  • Review prior class material
• Class 22 ():
  • Machine Learning (ML) CS Core - Session 3: Part 2 of 2
  • Q-Learning Project Demonstrations (cont.)
• Before class 23 ():
  • 4th Hour Robotics work
  • Review prior class material
• Class 23 ():
  • Q-Learning Project Demonstrations (cont.)
  • Machine Learning (ML) CS Core - Session 4: Part 1 of 2
• Before class 24 ():
  • 4th Hour Robotics work
  • Review prior class material
• Class 24 ():
  • Machine Learning (ML) CS Core - Session 4: Part 2 of 2
• Before class 25 ():
  • Review prior readings
• Class 25 ():
  • Brief description of Propositional Logic representation and reasoning.
  • Probabilistic Reasoning: Gibbs Sampling for Bayesian Network Reasoning
  • Gibbs sampling project work
    • Gibb sampling Bayes net code base
    • Begin HW7 exercise work in-class using Gibbs Sampling algorithm
• Before class 26 ():
  • Make sure you're caught up on readings.
• Class 26 ():
  • In-class applications of Gibbs sampling to demonstrate Bayesian network patterns of reasoning.
• Before class 27 ():
  • Bring 4th Hour Robotics projects to completion.
• Class 27 ():
  • Course Overview
  • Course Evaluations
  • Machine Learning exercise work together
• Before class 28 ():
  • Bring 4th Hour Robotics projects to completion.
• Class 28 ():
  • Demonstration of 4th Hour Robotics projects.