UIUC CS 598, Section EA
Decision Making Under Uncertainty
University of Illinois, Urbana-Champaign
Spring 2005

Index: Announcements Course Information Machinery Communication Textbook


Course Information

Lecture: 1 unit, TuTh 12:30-1:45PM, 1302 Siebel Center

Professor: Eyal Amir

Office: Siebel 3314
Phone: (217) 333-8756
email: eyal@cs.uiuc.edu
Office hours: Mon 8:30am-9:30am, Thu 3pm-4pm

Useful Information and Handouts

  • Syllabus & Important Dates
  • Handouts
  • Frequently Asked Questions

  • Course Descriptionm

    The course covers decision-making algorithms and systems in situations of partial knowledge. The focus is on decision making in dynamic situations when the state of the world is not completely observable, and that state has to be estimated using partial knowledge of the way the world evolves (in response to actions or otherwise) and the way world states and actions give rise to observations. Specifically, we will cover exact and approximate decision making using the following formal systems and guiding problems:
  • Basic methods: Fully observed systems: Markov Decision Problems (MDPs), Planning, Reinforcement learning
  • Basic methods: Unobserved systems: Unobserved MDPs, Conformant planning, Nondeterministic planning
  • FOCUS OF CLASS: Partial observability and sensing actions: Partially Observable MDPs (POMDPs), Planning with sensing, Reinforcement learning in POMDPs
  • Branch-out topic: Knowledge in logic (modalities) and probabilities (distributions over distributions)
  • Branch-out topic: Adversarial situations: Markov games, Games of partial information
  • Prerequisites

    [Books/Syllabus]  To catch up with knowledge required for the class, please read [Rusell & Norvig '03] (2nd edition) chapters 1-15, and [Cormen, Lasersson & Rivest '90] (or the newer edition).

    Machinery: Tasks and Coursework

    The course will consist of lectures by the teacher, lectures by students, and a final project, with the actual consistency depending on the number of students in the class. Due dates and recommended projects and papers for presentation are indicated in the syllabus.

    Presentations by students

    I expect students to present one or two papers during the course of the semester. They will select their paper/s from the list of recommended papers that is provided in the papers section of the syllabus. If a student wishes to present a paper that does not appear in this list, an approval from me (the instructor) will be required.

    Final project

    Students will also be expected to submit an initial project proposal and a more detailed proposal (after the first proposal was approved), according to the dates in the syllabus. Projects may have an implementation emphasis, a theoretical emphasis, or a combination (preferred, but not mandatory). I urge students to propose their own projects, but if they cannot find ones by themselves, then they may select projects from the project list provided in the projects section of the syllabus. In either case, the students should expect to meet with the instructor to present their initial proposal and their detailed proposal. Projects may be done by single students or in pairs. In the case of a pair of students, the project must be appropriately larger or more ambitious, and the contribution of each of the students clearly identifiable. In the conclusion of the class the students will present their projects in a poster session, and will also submit a technical paper describing the project (see syllabus for dates).

    The proposals and the final paper must be handed in during the class on the date indicated in the syllabus. Recognizing that students may face unusual circumstances and require some flexibility in the course of the quarter, each student will have a total of seven free late (calendar) days to use as s/he sees fit. Once these late days are exhausted, any paper or proposal turned in late will be penalized at the rate of 20% per late day (or fraction thereof). Under no circumstances will an assignment be accepted more than a week after its due date. Late days are from noon to noon.

    Late assignments should be turned in at the turn-in box outside my office. You must write the time and date of submission on the assignment. Alternatively, you can fax it to the course secretary (see the fax number above) or email it to eyal@cs.uiuc.edu.

    The final grade will be calculated using the following formula:
    0.25*technical paper + 0.25*poster + 0.3*presentation + 0.1*proposal1 + 0.1*proposal2
    Up to 3% extra credit may be awarded for class participation. Bonuses will be given for more ambitious, imaginative, and original work, but standard works that are correct and sufficient will still not be penalized and will receive full credit.


    I strongly encourage students to come to office hours instead of emailing questions to me. Also, as explained above, late assignments should be turned in at the turn-in box outside my office. You can also email these assignments to eyal@cs.uiuc.edu.

    Please do not e-mail me with grading questions. If you want me to explain why I took points off, you can talk to me after class or during office hours. If you want a regrade, please write an explanation and hand the assignment and the explanation to me during office hours or after class.

    Occasionally I may need to broadcast a message to entire class. I will do so over the WebBoard for the class.

    Textbooks and Papers Information

    The primary reading materials will be book chapters and papers as described in the syllabus. The main books people can use for this class are (each covers different material)

  • Michael Littman, Algorithms for Sequential Decision Making. 1996. Ph. D. thesis, Department of Computer Science, Brown University.
  • Richard S. Sutton and Andrew G. Barto, Reinforcement Learning: an introduction. 1998, MIT Press.
  • Dimitri P. Bertsekas and John Tsitsiklis, Neuro-Dynamic Programming. 1996, Athena Scientific.
  • Stuart Russell and Peter Norvig, Artificial Intelligence, a Modern Approach, Prentice Hall, 2nd ed., 2003.

  • Useful Links and Resources

  • Reid Simmons's class on AI planning
  • Son Tran's class on AI planning

  • Comments to Eyal Amir