CS 391 - Special Topic: Machine Learning
Course Information

Course Description

Machine Learning is a subfield of Artificial Intelligence that has a goal of building systems that can adapt to their environment or learn from experience.  Reinforcement learning is a specific approach to Machine Learning in which a software agent seeks to learn optimal behavior through interaction with a real or simulated environment.  The agent senses the current state of the environment, selects and performs an action, and consequently receives a numeric reward/penalty signal.  Through repeated experience of this process, the agent seeks to learn an action policy that maximizes its reward over time.  In this course, we will cover the reinforcement learning problem, dynamic programming, Monte Carlo methods, temporal-difference learning, eligibility traces, and function approximators.  Each student will then independently propose and complete a reinforcement learning programming project.


Reinforcement Learning: An Introduction
by Richard S. Sutton, Andrew G. Barto
MIT Press, 1998
ISBN: 0262193981


Todd Neller
Lecture: M,W,F 12:00-12:50, Glatfelter 203
Office: Glatfelter 209
Office Hours: M-F 10:00-10:50AM or by appointment. Note:  Generally, feel free to drop in if my office door is open.  If it is closed, I'm desperately seeking to keep on top of things and rabid attack ferrets may drop from the ceiling in my defense.
Phone: 337-6643


Your final grade will be determined from the following components:

You are responsible to know the material from each lecture and reading assignment before the start of the next class.  Homework is due at the beginning of lecture on the due date.  Late homework will not necessarily be accepted.  Homework submitted after the beginning of the following lecture will receive no credit.  Code that does not compile may not receive credit.  Class attendance and participation is required.  If you attend all classes and are willing to participate, you'll get 100% for this part of your grade.  Even if you know enough to give a particular lecture, please consider the value of helping your peers during in-class exercises

Honor Code

Honesty, Integrity, Honor.  These are more important than anything we will teach in this class.  Students can and are encouraged to help each other understand course concepts, but all graded work must be done independently. The work you submit (including both code and problem solving ideas expressed in the code) should be your independent work.  For detailed information about the honor code, see