**CPSC 533V: Learning to Move**
**[Weekly schedule](schedule.html)**
2024W1 (Sep-Dec 2024) - [UBC](http://www.ubc.ca) - [Department of Computer Science](http://cs.ubc.ca)
Course Description
========================================================================
This course is about learning to control the movement of humans, animals, and robots, with
application to robotics, physics-based character animation, biological motor control, and computer vision.
The bulk of the course will focus on reinforcement learning (RL), which has seen many significant advances over the past ten years
and is widely applicable to sequential decision-making problems of all kinds.
UBC undergraduates and other students who cannot by default register in the course
should complete the following [registration request form](https://forms.gle/xQy7sYdNPAQQDirT7).
We cannot promise that you will be able to register in the course - this will depend on
permissions from your program, your background, and space availability in the course.
_Topics_
: motion control problems, sequential decision problems and MDPs (Markov Decision Processes), RL fundamentals, dynamic programming,
known optimal solutions for linear dynamics, tabular methods, deep Q-learning,
policy gradient methods, common policy gradient algorithms (A2C, A3C, TRPO, PPO), common Q-learning algorithms (DDPG, SAC, TD3),
model-based RL, model-predictive control, behavior cloning, imitation learning, RL and representation learning, sim-to-real, RL frameworks,
advanced topics in RL
People
========================================================================
_Instructor_
: [Michiel van de Panne](http://cs.ubc.ca/~van), , ICCS x865.
_Teaching Assistants_
: Nicholas Ioannidis
Lectures
========================================================================
Monday | Wednesday | Thursday
-----------------------------------------|---------------------------------------------|--------------
LectureSWNG 408
1:30-3pm |Lecture SWNG 408
1:30-3pm | Office hours
ICCS x865, Mon 4-5
*Note:* There may be an eventual room change to a larger room, in which case it will be posted here.
At a minimum, the lectures will be recorded on Zoom until all registration issues are addressed.
See Piazza for the eventual links to these lectures.
I am also generally available at the end of class.
[Weekly schedule](schedule.html)
Sign up to [Piazza](http://piazza.com/ubc.ca/winterterm12024/cpsc533v), which will be used to handle many questions.
Evaluation
========================================================================
Component | Percentage
-------------------------------------|-------------------
Assignments (5) | 45%
[Readings, Presentations, Discussion](discussion.html) | 20%
[Project](project.html) | 35%
- [a1.pdf](a1.pdf)
- [a2](https://github.com/UBCMOCCA/CPSC533V_2024W1/tree/main) - due Fri Oct 18
Resources
========================================================================
- ["Reinforcement Learning: An Introduction" (Sutton & Barto, 2018)](http://incompleteideas.net/book/the-book-2nd.html)
- ["An Introduction to Deep Reinforcement Learning"](https://arxiv.org/pdf/1811.12560.pdf)
- [Open AI -- Spinning Up in Deep RL](https://spinningup.openai.com/en/latest/)
- more to be posted
Policies
========================================================================
_Illness_
: You are allocated three late days for the course to deal with unforeseen circumstances.
_Special Accomodations_
: Please contact the instructor.
-->