(ECE 271C) Dynamic Programming - Fall, 2017
This course will cover the fundamentals of dynamic programming which is a method for solving complex problems by breaking them down into simpler subproblems. Topics include deterministic and stochastic formulation of the principle of optimality, value and policy iteration, introduction to finite state Markov chains, partial state information problems, stochastic shortest path problems, infinite horizon problems, and introduction to approximate dynamic programming. Applications include inventory control, finance, routing, and sequential hypothesis testing.
Term: FAll Quarter, 2017
Lecture: Monday and Wednesday, 4:00-5:50, Phelps 1437
Office Hours: HFH 5161, Wednesday 1:00 -- 2:00, Friday 2:00 -- 2:50, or by email appointment
Text: Dynamic Programming and Optimal Control, Bertsekas, 3rd Edition, Volumes 1 and 2
Syllabus: [pdf]
Announcements:
- 10/02/2017: Welcome to the course website for ECE 271c - Dynamic Programming.
- 10/09/2017: I will be on travel 10/13-10/16. Office hours for Friday will be cancelled and we will make arrangements for Monday's class.
- 11/21/2017: There will be no class or office hours on Wednesday 11/22/2017.
Lectures
- 10/02/2017 - Lecture 1: [pdf]
- 10/04/2017 - Lecture 2: [pdf]
- 10/09/2017 - Lecture 3: [pdf]
- 10/11/2017 - Lecture 4: [pdf]
- 10/18/2017 - Lecture 5: [pdf]
- 10/23/2017 - Lecture 6: [pdf]
- 10/25/2017 - Lecture 7: [pdf]
- 10/30/2017 - Lecture 8: [pdf]
- 11/01/2017 - Lecture 9: [pdf]
- 11/06/2017 - Lecture 10: [pdf]
- 11/08/2017 - Lecture 11: [pdf]
- 11/13/2017 - Lecture 12: [pdf]
- 11/20/2017 - Lecture 13: [pdf]
- 11/27/2017 - Lecture 14: [pdf]
- 11/29/2017 - Lecture 15: [pdf]
- 12/04/2017 - Lecture 16: [pdf]
- 12/06/2017 - Lecture 17: [pdf]
Homeworks
- Homework 1 (due 10/11/2017 in class): [pdf] [sols]
- Test Case 1: [mat] Answer = [2 2 1 2 3]
- Test Case 2: [mat] Answer = [1 1 2 1 5 2]
- Test Case 3: [mat] Answer = [1 1 3 4 2 6 2]
- Homework 2 (due 10/23/2017 in class): [pdf] [sols]
- Homework 3 (due 11/01/2017 in class): [pdf] [sols]
- Homework 4 (due 11/13/2017 in class, 11/14/2017 in office): [pdf] [sols]
- Homework 5 (due 11/29/2017 in class, 12/4/2017 in office): [pdf]
- Grade Policy:
- Homework Policy: Collaboration is fine, however each person must submit their own copy of the homework. To complete the HW you can use lecture notes and the text. You are not permitted to use alternative sources of information such as the internet or alternative texts/lecture notes.