In this course, we will look into how Deep Learning is used for 3D Computer Vision. Specifically, we first look into the basics of Deep Learning, then into epipolar geometry, the math governing the geometry behind two cameras and a point. We will then integrate Deep Learning with epipolar geometry, through learned local features and Deep Learning on point clouds. We will also look into how neural fields can be used in this context, including NeRF and more recent Gaussian Splatting. We will also see how recent Generative models, such as diffusion-based models integrate into the 3D Computer Vision landscape.
| Instructor: Kwang Moo Yi (kmyi@cs.ubc.ca | https://www.cs.ubc.ca/~kmyi) |
TA: TBA
Lectures will be in-person at ORCH-Floor 3-Room 3005, Mon/Wed 10:30 - 12:00
Note: This year I will try remove some content to buy space for Diffusion models and Gaussian Splats.
| Date | Topic (Slides) | Description | Reading | Notes |
|---|---|---|---|---|
| Introduction | Introduction and preview | |||
| Deep learning basics | Deep learning basics 1 | |||
| Deep learning basics | Deep learning basics 2 | |||
| Deep learning basics | Deep learning basics 3 | A1 Posted | ||
| Epipolar geometry | Multiple-view geometry 1 | |||
| Epipolar geometry | Multiple-view geometry 2 | A1 Due | ||
| Epipolar geometry | Multiple-view geometry 3 | A2 Posted | ||
| Local features | Local features 1 | |||
| Local features | Local features 2 | A2 Due | ||
| Local features | Local features 3 | |||
| Point cloud networks | Point cloud network architectures | |||
| Learned image correspondences | Image correspondences with point cloud networks | Project Proposals Due | ||
| Point clouds generative models | Generative models with point clouds | A3 Posted, Review Paper Selection Due | ||
| Neural Fields | Compositional Pattern Producing Networks | |||
| Neural Fields | Implicit Shape Modeling | |||
| Neural Radiance Fields | Neural Rendering | A3 Due | ||
| Neural Radiance Fields | More Neural Radiance Fields | A4 Posted | ||
| Seminar | Paper Reviews and Seminar 1 | |||
| Seminar | Paper Reviews and Seminar 2 | A4 Due | ||
| Seminar | Paper Reviews and Seminar 3 | Project Progress Report Due | ||
| Project Presentations | Term Project Presentations 1 | Paper Reviews Due | ||
| Project Presentations | Term Project Presentations 2 | |||
| Project Presentations | Term Project Presentations 3 | Project Report Due |
Any components that a misses due to legitimate reasons (e.g., health reasons and attending conferences) will be removed with from the total (e.g., your total will be now computed out of 85% rather than the full 100%). You may be asked to provide supporting documentation.
The academic enterprise is founded on honesty, civility, and integrity. As members of this enterprise, all students are expected to know, understand, and follow the university policies and codes of conduct regarding academic integrity. At the most basic level, this means submitting only original work done by you and acknowledging all sources of information or ideas, and attributing them to others as required. This also means you should not cheat, copy, or mislead others about what is your work; nor should you help others to do the same. For example, it is prohibited to: share your past assignments and answers with other students; work with other students on an assignment when an instructor has not expressly given permission; or spread information through word of mouth, social media, or other channels that subverts the fair evaluation of a class exercise, or assessment. Violations of academic integrity (i.e., misconduct) lead to the breakdown of the academic enterprise, and therefore serious consequences arise and harsh sanctions are imposed. For example, incidences of plagiarism or cheating may result in a mark of zero on the assignment or exam and more serious consequences may apply if the matter is referred to the President’s Advisory Committee on Student Discipline. Careful records are kept in order to monitor and prevent recurrences.
UBC is trying in earnest to encourage diversity and alleviate biases and inequities to which some members of its community are subjected; this includes, for example, UBC’s Indian Residential School History and Dialogue Center, and well as the Computer Science Department’s various programs described on its Diversity in CS webpage. I try to act reasonably free of bias; for example, I do not view sexual orientation or gender as set in stone from birth or as classified by some fixed, finite set of choices; I try to use language accordingly. I will undoubtedly goof upon occasion, and I welcome feedback on these and related matters.
UBC provides resources to support student learning and to maintain healthy lifestyles but recognizes that sometimes crises arise and so there are additional resources to access including those for survivors of sexual violence. UBC values respect for the person and ideas of all members of the academic community. Harassment and discrimination are not tolerated nor is suppression of academic freedom. UBC provides appropriate accommodation for students with disabilities and for religious and cultural observances. UBC values academic honesty and students are expected to acknowledge the ideas generated by others and to uphold the highest academic standards in all of their actions. Details of the policies and how to access support are available here: https://senate.ubc.ca/policies-resources-support-student-success
In-person, on campus activities may need to be cancelled due to issues such as weather conditions (e.g., snow). The most up-to-date information about cancellations will be posted on ubc.ca. Please check ubc.ca often during times when an extreme weather event could disrupt our course activities. If in-person classes or exams are cancelled, the following contingency plans will take effect. The uncertainty that comes with extreme weather events can be stressful. Rest assured I will be flexible with assignment deadlines and communicate with you as early as I can. I will try to communicate with you about weather-related class cancellations through Canvas announcements. Here is what you can expect in the event an in-person class session, quiz, or exam is cancelled:
If in-person activities are cancelled due to weather or other environmental conditions, class will be held online. The Zoom link will be posted on Canvas. For those unable to participate in an online class on short notice, I will provide a lecture recording that is posted to Canvas.
If weather impacts the midterm, it will be dropped and the weight will be redistributed to other course components and the course total will be computed without the midterm component.
If you are registered to write exams at the Centre for Accessibility, I encourage you to reach out to your CFA advisor well in advance to discuss the weather contingency plan for this course.
If you have any questions or concerns about this weather contingency plan, please come talk to me. Discussing any issues prior to the cancellation is helpful so we can work out a plan in advance.