Computer vision, broadly speaking, is a research field aimed to enable computers to process and interpret visual data (namely in the form of images and video), as sighted humans can. It is one of the most exciting areas of research in computing science and among the fastest growing technologies in today’s industry. This course provides an introduction to the fundamental principles and applications of computer vision, including image formation, sampling and filtering, color analysis, single and multi-image geometry, feature detection and matching, stereo imaging, motion estimation, segmentation, image classification and object detection. We’ll study basic methods and application of these concepts to a variety of visual tasks.
| Instructor: Kwang Moo Yi (kmyi@cs.ubc.ca | https://www.cs.ubc.ca/~kmyi), Matthew Brown (https://mattabrown.github.io) |
TA: Yeojun
TA: Jeff (Yang-Che)
TA: Shunsuke
TA: Bicheng
SPPH-Floor B1-Room B151https://piazza.com/class/m4rloflhm2n9e/ (sign up)
MATH 200, MATH 221 and either (a) CPSC 221 or (b) CPSC 260, EECE 320.
Recommended but NOT required.
Assignments are to be done individually by each student. We will be actively looking for cases of academic misconduct (see below). We will have zero-tolerance on any case, and will not make any distinctions between those who helped or those who received help. And frankly, it’s not worth it!
| Assignment | Available | Due |
|---|---|---|
| Assignment 0: Introduction to Python for Computer Vision (optional) | Jan 6 | Jan 15 (optional) |
| Assignment 1: Image Filtering and Hybrid Images | Jan 13 | Jan 30 |
| Assignment 2: Scaled Representations, Face Detection and Image Blending | Jan 27 | Feb 13 |
| Assignment 3: Texture Synthesis | Feb 12 | Mar 6 |
| Assignment 4: RANSAC and Panorama Stitching | Mar 4 | Mar 20 |
| Assignment 5: Stereo and Optical Flow | Mar 18 | Apr 3 |
| Assignment 6: Deep Learning | Apr 1 | Apr 10 |
Every student is allotted three ``late days’’, which allow assignments to be handed in late without penalty on three days or parts of days during the term. The purpose of late days is to allow students the flexibility to manage unexpected obstacles to coursework that arise during the course of the term, such as travel, moderate illness, conflicts with other courses, extracurricular obligations, job interviews, etc. Thus, additional late days will NOT be granted except under truly exceptional circumstances. If an assignment is submitted late and a student has used up all of her/his late days, 25% will be deducted for every day the assignment is late. (e.g., an assignment 2 days late and graded out of 100 points will be awarded a maximum of 50 points.) How late does something have to be to use up a late day? A day is defined as a 24-hour block of time beginning at 10 minutes past 11:59pm on the day an assignment is due. To use a late day, write the number of late days claimed on the first page of your assignment. Examples: Handing in an assignment at 1am the night assignment is due will use up 1 late day. Handing in an assignment at 10:15am the morning after it is due similarly consumes one late day. Handing in an assignment at 10:15am on the following day consumes two late days.
The exact schedule may change according to course progress. Notes are from Matthew Brown.
| Date | Topic (Slides) | Description | Reading | Notes |
|---|---|---|---|---|
| Jan 6 | Introduction (Updated 25’) | Introduction to Computer Vision and course logistics | ||
| Jan 8 | Image Formation (Updated 25’) | Light, Reflectance, Cameras and Lenses, Pinhole, Perspective, Orthographic projection | Szeliski 2.2 | Notes |
| Jan 13 | Image Filtering 1 (Updated 25’) | Image as a function, Image Transformations, Linear filters, Correlation and Convolution | Szeliski 3.1-3.2 | Notes |
| Jan 15 | Image Filtering 2 (Updated 25’) | Gaussian, Pillbox filter, Separability, Low Pass / High Pass Filtering | Szeliski 3.2 | Notes |
| Jan 20 | Image Filtering 3 (Updated 25’) | Non-Linear Filters, Bilateral Filters, Speeding up Convolution, Fourier Representation | Szeliski 3.3 | Notes, Quiz 1 |
| Jan 22 | Sampling (Updated 25’) | Sampling Theory, Bandlimited Signal, Nyquist Rate, Aliasing, Color Filter Arrays, Demosaicing | Szeliski 2.3 | Notes |
| Jan 27 | Template Matching (Updated 25’) | Digital Imaging Pipeline continued, Template Matching, Correlation, Normalised Correlation, SSD | Szeliski 9.1 (up-till 9.1.1) | Notes |
| Jan 29 | Scaled Representations (Updated 25’) | Gaussian Pyramid, Laplacian Pyramid, Pyramid Blending, Multi-Scale Template Matching and Detection | Szeliski 3.5 | Notes, Quiz 2 |
| Feb 3 | Edge Detection (Updated 25’) | Image Derivatives, Edge Filtering, 2D Gradient, Canny Edge Detection, Image Boundaries | Szeliski 7.2 | Notes |
| Feb 5 | Corner Detection (Updated 25’) | Image Structure, Corner Detection, Autocorrelation, Harris, Scale Selection, DoG | Szeliski 7.1 | Notes |
| Feb 10 | Texture (Updated 25’) | Texture Representation, Filter Banks, Textons, Non-Parameteric, Texture Synthesis | Szeliski 10.5 | |
| Feb 12 | Midterm Prep (See Canvas for slides) | Quiz 3 | ||
| Feb 17 | Midterm Break | No class | ||
| Feb 19 | Midterm Break | No class | ||
| Feb 24 | Midterm | In-class midterm | ||
| Feb 26 | Local Image Features (Updated 25’) | Correspondence, Invariance, Geometric, Photometric, SIFT, Object Instance Recognition | Szeliski 7 | |
| Mar 3 | Planar Transformation (Updated 25’) | 2D Transformations, Similarity, Euclidan, Affine, Homography, Robust Estimation, RANSAC | Szeliski 2.1, 8.1 | Notes |
| Mar 5 | Hough Transform (Updated 25’) | Hough Transform, Line Detection, Transformation Space Voting | Szeliski 7.4 | Quiz 4 |
| Mar 10 | Stereo (Updated 25’) | Epipolar Geometry, Rectification, Disparity, Block Matching, Occlusions, Ordering Constraints | Szeliski | Notes, |
| Mar 12 | Optical Flow (Updated 25’) | Brightness Constancy, Optical Flow Constraint, Aperture Problem, Lucas-Kanade, Horn-Schunck | Szeliski 9.3 | Notes |
| Mar 17 | Multiview Reconstruction (Updated 25’) | Multiview Matching, Bundle Adjustment, Pose Estimation, Triangulation, Image Alignment and 3D Reconstruction | Szeliski 11.4, 12.3-12.4 | Notes |
| Mar 19 | Visual Classification 1 (Updated 25’) | Instance and Category Recognition, Viusal Words, Bag of Words, Support Vector Machines | Szeliski 11.4, 12.3-12.4, 9.3, 5.1-5.2 | Quiz 5 |
| Mar 24 | Visual Classification 2 (Updated 25’) | Linear Classification, Nearest Neighbour, Nearest Mean, Bayesian Classifiers, One-Hot Regression, Regularisation, SGD, Momentum | Szeliski 11.4, 12.3-12.4, 9.3, 5.1-5.2 | Notes |
| Mar 26 | Neural Networks 1 | Activation Functions, Softmax, Cross Entropy, Update rules, Perceptron, 2-layer Net, Gradients, Backpropagation | Szeliski 5.1.3, 5.3-5.4 | Notes |
| Mar 31 | Neural Networks 2 | Linear layer backward pass, Convolutional Neural Networks, Strided convolution, Max Pooling, Deep Learning, AlexNet, VGG | Notes | |
| Apr 2 | Neural Networks 3 | Weight Initialization, Normalization, Preventing Overfitting | ||
| Apr 7 | Final Prep | Quiz 6 |
Any components that a misses due to legitimate reasons (e.g., health reasons and attending conferences) will be removed with from the total (e.g., your total will be now computed out of 85% rather than the full 100%). You will be asked to provide supporting documentation.
Despite best efforts, sometimes miss-grading does happen. All grade disputes and re-grading must be brought to instructor’s or TA’s attention within 1 week of the grade being released. All such requests must be done through a Piazza as a direct post to the instructors. You must clearly identify the issues and describe why you believe re-grading is warranted.
The academic enterprise is founded on honesty, civility, and integrity. As members of this enterprise, all students are expected to know, understand, and follow the university policies and codes of conduct regarding academic integrity. At the most basic level, this means submitting only original work done by you and acknowledging all sources of information or ideas, and attributing them to others as required. This also means you should not cheat, copy, or mislead others about what is your work; nor should you help others to do the same. For example, it is prohibited to: share your past assignments and answers with other students; work with other students on an assignment when an instructor has not expressly given permission; or spread information through word of mouth, social media, or other channels that subverts the fair evaluation of a class exercise, or assessment. Violations of academic integrity (i.e., misconduct) lead to the breakdown of the academic enterprise, and therefore serious consequences arise and harsh sanctions are imposed. For example, incidences of plagiarism or cheating may result in a mark of zero on the assignment or exam and more serious consequences may apply if the matter is referred to the President’s Advisory Committee on Student Discipline. Careful records are kept in order to monitor and prevent recurrences.
UBC is trying in earnest to encourage diversity and alleviate biases and inequities to which some members of its community are subjected; this includes, for example, UBC’s Indian Residential School History and Dialogue Center, and well as the Computer Science Department’s various programs described on its Diversity in CS webpage. I try to act reasonably free of bias; for example, I do not view sexual orientation or gender as set in stone from birth or as classified by some fixed, finite set of choices; I try to use language accordingly. I will undoubtedly goof upon occasion, and I welcome feedback on these and related matters.
UBC provides resources to support student learning and to maintain healthy lifestyles but recognizes that sometimes crises arise and so there are additional resources to access including those for survivors of sexual violence. UBC values respect for the person and ideas of all members of the academic community. Harassment and discrimination are not tolerated nor is suppression of academic freedom. UBC provides appropriate accommodation for students with disabilities and for religious and cultural observances. UBC values academic honesty and students are expected to acknowledge the ideas generated by others and to uphold the highest academic standards in all of their actions. Details of the policies and how to access support are available here: https://senate.ubc.ca/policies-resources-support-student-success
In-person, on campus activities may need to be cancelled due to issues such as weather conditions (e.g., snow). The most up-to-date information about cancellations will be posted on ubc.ca. Please check ubc.ca often during times when an extreme weather event could disrupt our course activities. If in-person classes or exams are cancelled, the following contingency plans will take effect. The uncertainty that comes with extreme weather events can be stressful. Rest assured I will be flexible with assignment deadlines and communicate with you as early as I can. I will try to communicate with you about weather-related class cancellations through Canvas announcements. Here is what you can expect in the event an in-person class session, quiz, or exam is cancelled:
If in-person activities are cancelled due to weather or other environmental conditions, class will be held online. The Zoom link will be posted on Canvas. For those unable to participate in an online class on short notice, I will provide a lecture recording that is posted to Canvas.
If weather impacts the midterm, it will be dropped and the weight will be redistributed to other course components and the course total will be computed without the midterm component.
If you are registered to write exams at the Centre for Accessibility, I encourage you to reach out to your CFA advisor well in advance to discuss the weather contingency plan for this course.
If you have any questions or concerns about this weather contingency plan, please come talk to me. Discussing any issues prior to the cancellation is helpful so we can work out a plan in advance.