UIUC CS 598, Section EA
Decision Making Under Uncertainty
University of Illinois, Urbana-Champaign
Spring 2005
Tentative Course Syllabus

Approximate Schedule (updated 1/21/2005)

Session Date Topic Readings Sample Application
Reading
Assignment
1
Jan 18
Introduction to Decision-Making Mobile Robotics:
inverted helicopter video (by Andrew Ng), retiarius video (by Andrew Ng)
Signup for paper presentations begins
2
Jan 20
Conformant, Nondeterministic Planning
[Cimatti & Roveri; JAIR 2000], Slides lec.2 (based on slides by Jose Ambite, Paolo Traverso, and Rune Jensen)
   
3
Jan 25
Planning using OBDDs
[Bertoli etal; IJCAI 2001], Slides lec.3 (using slides by Jose Ambite, Paolo Traverso, and Rune Jensen)
   
4
Jan 27
Planning with Sensing
[Bertoli & Pistore; ICAPS 2004], Slides lec.4 (using some slides by Son Tran, Chitta Baral)
   
5
Feb 1
Planning with Sensing
[Bertoli & Pistore; ICAPS 2004], Slides lec.4 (using some slides by Son Tran, Chitta Baral)
   
6
Feb 3
Markov-Decision Problems (MDPs)
[Littman; Brown U. thesis 1996] ch. 2, Slides lec.5 (using slides by Craig Boutilier)  
   
7
Feb 8
Markov-Decision Problems (MDPs)
[Littman; Brown U. thesis 1996] ch. 2, Slides lec.7 (using slides by Craig Boutilier)  
   
8
Feb 10
Reinforcement Learning
[Kaelbling & Littman; JAIR '96], Slides lec.8 (using slides by Jeremy Wyatt)
   
9
Feb 15
Approximate Value Functions
[Sutton & Barto '98] ch.8 , Slides lec.9 (using slides by Jeremy Wyatt, Ron Parr, Craig Boutilier, and Eduardo Alonso)
   
10
Feb 17
Partially Observable MDPs (POMDPs)
[Littman; Brown U. thesis 1996] ch. 6-7, Slides lec.10 (using slides by Craig Boutilier)
  Signup for paper presentations ends
11
Feb 22
Partially Observable MDPs (POMDPs)
[Littman; Brown U. thesis 1996] ch. 6-7, Slides lec.10 (using slides by Craig Boutilier)
   
12
Feb 24
Paper presentation: MDPs   Alexander Sorokin
13
Mar 1
Paper presentation: Information Gathering   Hector Gonzalez
14
Mar 3
NO CLASS
 
   
15
Mar 8
Paper presentation: Exploration   Proposal 1 due
Michael Simon
16
Mar 10
Factored Planning    
17
Mar 15
Paper presentation: approximate value   Vladimer Vladimerou
18
Mar 17
Subsumption-Architecture-style Synthesis
&
  Extended proposal due
19
Mar 29
Paper presentation: Factored MDPs   Arkady Epshteyn
20
Mar 31
Decision Making presentation: Prof. Ali Abbas (Gen Eng, UIUC)
 
   
21
Apr 5
Paper presentation: POMDPs   Hannaneh Hajishirzi
22
Apr 7
Projects mid-semester review
 
  3-5 min. presentations in class
23
Apr 12
Paper presentation: POMDPs   Geoff Levine
24
Apr 14
Paper presentation: POMDPs   Ben Liebald
25
Apr 19
Paper presentation: First-Order MDPs   Chi Trinh
26
Apr 21
Paper presentation: open
 
   
27
Apr 26
Paper presentation: inverse RL   Hossein Mobahi
28
Apr 28
Paper presentation: Risk-sensitive planning   Jeff Pasternack
29
May 3
Posters session (5pm-7pm)
 
  final project submission

Papers for Presentations

 
Number Topic (PO = Partially Observable; PL = Planning; ST = Stochastic; SM = Semantics; RL = Reinforcement Learning) Paper/s Sample Application
Reading
Presenter
1
Information Gathering TBA
TBA
Hector Gonzalez
2
Sensing Actions (PO, PL) TBA
TBA
 
3
Sensing Actions (PO, PL) TBA
TBA
 
4
Triggered Actions TBA
TBA
 
5
Formalisms for Planning w/Sensing (PL) TBA
TBA
James Davidson
6
planning w/sensing (SM) TBA
TBA
 
7
POMDPs (RL;ST) TBA
TBA
 
8
Exploration (ST) TBA
TBA
Michael Simon
9
Utility elicitation and POMDPs (PL;ST) TBA
TBA
 
10
POMDPs models customers TBA
TBA
 
11
POMDP approximation TBA
TBA
 
12
POMDP approximation TBA
TBA
Geoff Levine
13
POMDP approximation TBA
TBA
 
14
POMDP approximation    
15
First-Order MDPs   Chi Trinh
16
Factored MDPs    
17
Monitoring POMDPs    
18
MDPs    
19
MDPs    
20
MDPs (survey)    
21
Reinforcement Learning (survey)    
22
MDPs & POMDPs (survey)    
23
Programmable RL agents   Alexander Sorokin
24
Inverse RL   Hossein Mobahi
25
Function Approximation in MDPs   Vladimer
26
Reward Shaping in MDPs    
27
Factored MDPs   Arkady Epshteyn
28
POMDPs as DBNs   Hannaneh Hajishirzi
29
Relational MDPs    
30
Approximating POMDPs    
31
POMDPs    
32
Approximating POMDPs    
33
Approximating POMDPs   Ben Liebald
34
Approximating POMDPs    
35
Planning with Nondeterminism and Sensing    
36
Planning with Sensing    
37
Planning with Sensing    
38
Planning with Sensing: unifying view    
38
Risk-sensitive planning   Jeff Pasternack

Possible Projects

Number Topic Presenter
1
Kriegspiel game player  
2
First-Order POMDPs  
3
Exact Factored MDPs  
4
Conformant planning using Logical Filtering  
5
Activity detection using filtering (any method)  
6
SLAM2.0 improved and applied to a mobile robot  
7
Poker Playing  
8
Bridge Player  
9
First-Order Factored Planning  
10
Adventure-Game Exploration using Commonsense knowledge  
11
Approximate Deterministic POMDPs via Logical Filtering  
12
Probabilistic Resolution in Dynamic Bayesian Networks  
13
POMDPs approximated via DBNs  
14
Robot control using a POMDP  
15
First-Order Reinforcement Learning  
16
Reshaping rewards using commonsense knowledge  
17
Partially observable LSA-based robot control architecture  
18
Reinforcement learning in deterministic domains  
19
Controlling a Levitating Robot  
20
Robot localization for a basketball game  
21
LSA-based control system for a robotic arm  
21
Reward shaping for POMDPs  


Comments to Eyal Amir