Robot Tour Guide Examples 

 MDP Grid World

Problems With Conventional Planning 

For this problem (and only this problem) assume actions are stochastic in a way that is different than described in 4. MDP Gridworld.

Instead of an action north possibly going east or west, an action north will possibly go northeast or northwest (i.e. to the diagonal squares).

Likewise for the other directions e.g. an action west will possibly go west, northwest or southwest (i.e. to the diagonals).

Stochastic actions are as in 4. MDP Grid World.

An action North moves North with 80% chance otherwise East with 10% chance or West with 10% chance. Likewise for the other directions.

 MDP And Costs

Value Iteration

Value Iterations And Policy 

MDP Conclusion 

POMDP 

go south first to read the sign

Readings on Planning under Uncertainty

AIMA: Chapter 17

Further Study

Charles Isbell and Michael Littmann’s ML course:

• Markov Decision Processes
• Reinforcement Learning

Peter Norvig and Sebastian Thrun’s AI course:

• Reinforcement Learning

 http://aima.cs.berkeley.edu/