CS 598, Section EA
Decision Making Under Uncertainty
University of Illinois, Urbana-Champaign
Lecture: 1 unit, TuTh 12:30-1:45PM, 1302 Siebel Center
Professor: Eyal Amir
- Office: Siebel 3314
- Phone: (217) 333-8756
- email: email@example.com
- Office hours: Mon 8:30am-9:30am, Thu 3pm-4pm
Useful Information and Handouts
- Syllabus & Important Dates
- Frequently Asked Questions
The course covers decision-making algorithms and systems in situations
of partial knowledge. The focus is on decision making in dynamic
situations when the state of the world is not completely observable, and
that state has to be estimated using partial knowledge of the way the
world evolves (in response to actions or otherwise) and the way world states
and actions give rise to observations.
Specifically, we will cover exact and approximate decision making using
the following formal systems and guiding problems:
- Basic methods: Fully observed systems: Markov Decision Problems (MDPs), Planning, Reinforcement learning
- Basic methods: Unobserved systems: Unobserved MDPs, Conformant planning, Nondeterministic planning
- FOCUS OF CLASS: Partial observability and sensing actions: Partially Observable MDPs (POMDPs), Planning with sensing, Reinforcement learning in POMDPs
- Branch-out topic: Knowledge in logic (modalities) and probabilities (distributions over distributions)
- Branch-out topic: Adversarial situations: Markov games, Games of partial information
To catch up with knowledge required for the class, please read
[Rusell & Norvig '03] (2nd edition) chapters 1-15, and [Cormen, Lasersson & Rivest '90] (or the newer edition).
- Familiarity with the basic concepts of logic and probability theory.
- Knowledge of basic computer science principles and skills.
- Knowledge of basic artificial intelligence problems and principles (at least at the level of CS440, and preferably at the level of CS498ea).
- Mathematical ability and the ability to understand and analyze
fairly complicated algorithms and data structures. (CS473 is
sufficient but not necessary.)
|Machinery: Tasks and Coursework
The course will consist of lectures by the teacher, lectures by
students, and a final project, with the actual consistency depending
on the number of students in the class.
Due dates and recommended projects and papers for presentation
are indicated in the syllabus.
Presentations by students
I expect students to present one or two papers during the
course of the semester. They will select their paper/s from the list
of recommended papers that is provided in the
papers section of the syllabus.
If a student wishes to present a paper that does not
appear in this list, an approval from me (the instructor) will be required.
Students will also be expected to submit an initial project proposal
and a more detailed proposal (after the first proposal was approved),
according to the dates in the syllabus. Projects may have an
implementation emphasis, a theoretical emphasis, or a combination
(preferred, but not mandatory). I urge students to propose their own
projects, but if they cannot find ones by themselves, then they may
select projects from the project list provided in the
projects section of the syllabus.
In either case, the students should expect to meet with the instructor
to present their initial proposal and their detailed
proposal. Projects may be done by single students or in pairs. In the
case of a pair of students, the project must be appropriately larger
or more ambitious, and the contribution of each of the students
clearly identifiable. In the conclusion of the class the students
will present their projects in a poster session, and will also submit
a technical paper describing the project (see syllabus for dates).
The proposals and the final paper must be handed in during the class
on the date indicated in the syllabus. Recognizing that students may
face unusual circumstances and require some flexibility in the course
of the quarter, each student will have a total of seven free
late (calendar) days to use as s/he sees fit. Once these late days
are exhausted, any paper or proposal turned in late will be penalized at the
rate of 20% per late day (or fraction thereof). Under no
circumstances will an assignment be accepted more than a week after its
due date. Late days are from noon to noon.
Late assignments should be turned in at the turn-in box outside my
office. You must write the time and date of submission on
the assignment. Alternatively, you can fax it to the course secretary
(see the fax number above) or email it to
The final grade will be calculated using the following formula:
0.25*technical paper + 0.25*poster + 0.3*presentation + 0.1*proposal1 + 0.1*proposal2
Up to 3% extra credit may be awarded for class participation.
Bonuses will be given for more ambitious, imaginative, and original work, but standard works that are correct and sufficient will still not be penalized and will receive full credit.
I strongly encourage students to come to office hours instead
of emailing questions to me. Also, as explained above, late
assignments should be turned in at the turn-in box outside my
office. You can also email these assignments to firstname.lastname@example.org.
Please do not e-mail me with grading questions. If you want me to explain
why I took points off, you can talk to me after class
or during office hours. If you want a regrade, please write an
explanation and hand the assignment and the explanation to me
during office hours or after class.
Occasionally I may need to broadcast a message to entire class. I will do
so over the WebBoard for the class.
|Textbooks and Papers Information|
The primary reading materials will be book chapters and papers as described in the syllabus.
The main books people can use for this class are (each covers different material)
Algorithms for Sequential Decision Making. 1996. Ph. D. thesis, Department of Computer Science, Brown University.
Richard S. Sutton and Andrew G. Barto,
Reinforcement Learning: an introduction. 1998, MIT Press.
Dimitri P. Bertsekas and John Tsitsiklis,
Neuro-Dynamic Programming. 1996, Athena Scientific.
Stuart Russell and Peter Norvig,
Artificial Intelligence, a Modern Approach, Prentice Hall, 2nd ed., 2003.
|Useful Links and Resources|
Reid Simmons's class on AI planning
Son Tran's class on AI planning