Announcements
- June 29, 2021 Assignment files have been taken off the website. If you need to access them for whatever reason, please contact me.
- June 12, 2021 Assignment 6 has been released and is due 11:55pm on Friday, June 18, 2021. Start early!!
- June 7, 2021 Final exam will be on Wednesday, June 23, 2021.
- June 3, 2021 Assignment 5 has been released and is due 9:25am on Friday, June 11, 2021. Start early!!
- May 27, 2021 Assignment 4 has been released and is due 9:25am on Monday, June 7, 2021. Start early!!
- May 20, 2021 Assignment 3 has been released and is due 9:25am on Friday, May 28, 2021. Start early!!
- May 16, 2021 Assignment 2 has been released and is due 9:25am on Monday, May 24, 2021. Start early!!
- May 10, 2021 Assignment 1 has been released and is due 9:25am on Monday, May 17, 2021. Start early!!
- May 9, 2021 Join the first lecture @ 9:30AM tomorrow here.
This Semester
Instructor
Lectures Sections (beginning May 10, 2021)
- 9:30am - 12:00 pm (Monday/Wednesday/Friday) online through Zoom.
Office hours
- Instructor: Tuesdays and Thursdays, 3:30pm-5:00pm
- TAs:
- See the calendar below and the Zoom tab on Canvas for the most accurate schedule. The TA office hours will be delivered through Zoom.
Tutorials (beginning Monday, May 10, 2021)
- There are 8 tutorials each week.
- The tutorials start on May 10th.
- T1A: Mon/Wed 1:30-2:30 pm.
- T1B: Mon/Wed 2:30-3:30 pm.
- T1C: Mon/Wed 3:30-4:30 pm.
- T1D: Mon/Wed 12:30-1:30 pm.
- The tutorials will be delivered through Zoom.
Teaching assistants
Services
- Piazza for course-related questions and HW solutions (accessible through Canvas).
- Zoom for lecture presentations and office hours.
- Gradescope for homework assignments (accessible through Canvas).
- Google Calendar for course schedule (see below).
Course Calendar
- I recommend that you subscribe to this calendar to always have the up-to-date schedule.
Registration
Undergraduate and graduate students from any department are welcome to take the course. Undergraduate students should enroll in CPSC 340 while graduate students should enroll in CPSC 532M (which has an extra small project component -- not offered in summers). Below are more details on registration for each course:
- The majority of the seats in 340 are reserved for UBC computer science majors. For other students, to enroll in the course you need to sign up for the wait list. Note that last year all students on the wait list were ultimately accepted into the course (but we did not have room for auditors.)
- Two-thirds of the seats in 532M will initially be reserved for UBC computer science grad students, while the remaining seats are open to graduate students from any department. Note that you will likely get a warning about prerequisities when you register (which you can ignore), but depending on your department you may later be contacted and asked to show how you satisfy the relevant prerequisites. As with 340, if 532M becomes full then signing up for the waiting list is the only way to enroll in the course. Last year all students on the wait list were ultimately accepted into the course.
Starting in the first week of classes, we will have weekly tutorials run by the TAs. These will do things like go through provided assignment code, review background material, review big concepts, and/or do exercises. You can register for particular tutorial sections if you want to save a seat at a particular time, but note that you do not need to register in a tutorial section.
Prerequisites
- Basic algorithms and data structures (CPSC 221, or both of CPSC 260 and EECE 320 as well as one of CPSC 210, EECE 201, or EECE 309).
- Linear algebra (one of MATH 152, 221, or 223).
- Probability (one of STAT 241, STAT 251, ECON 325, ECON 327, MATH 302, STAT 302, or MATH 318).
- Multivariate calculus (one of MATH 200, 217, 226, 253, or 263).
Undergraduate and graduate students from any department are welcome to take the class, provided that they satisfy the prerequisites. If you do not satisfy the exact prerequisites but would still like to enroll in the class, see
here. For graduate students from outside the CS department, see
here.
Textbook
There is no required textbook for the class. A introductory book that covers many (but not all) the topics we will discuss is the Artificial Intelligence book of Rusell and Norvig (AI:AMA) or the Artificial Intelligence book of Poole and Mackworth (you may need these for other classes). More advanced books include The Elements of Statistical Learning (ESL) by Hastie et al., Murphy's Machine Learning: A Probabilistic Perspective (ML:APP) which can be accessed through the library here, and Bishop's Pattern Recognition and Machine Learning (PRML). For books with a bigger focus on data mining, see Introduction to Data Mining (IDM) and Mining of Massive DataSets.
Related Courses
Related courses in statistics include:
STAT 305,
STAT 306,
STAT 406,
STAT 460,
STAT 461 (as well as
EOSC 510). A discussion of the difference between CPSC 340 and these various STAT classes written by a former student (Geoff Roeder) is available
here.
Grading
- Assignments 30%
- Midterm 20%
- Final 50%
List of topics
We will roughly cover the following topics:
- Data representation and summarization.
- Supervised learning with frequencies and distances.
- Data clustering, outlier detection, and association rules.
- Linear prediction, regularization, and kernels.
- Latent-factor models and collaborative filtering.
- Neural networks and deep learning.
Lectures
| Date |
Slides |
Related Readings and Links |
Notes |
Notebooks |
| Mon May 10 |
Motivation and Syllabus (UPDATED)
|
What is Machine Learning? ·
Machine Learning on Wikipedia ·
Rise of the Machines ·
Artificial Intelligence-The Revolution Hasn't Happened Yet ·
Machine Learning: The Great Stagnation ·
Stop Calling Everything AI, Machine-Learning Pioneer Says ·
|
|
|
|
Exploratory Data Analysis (UPDATED)
|
Gotta Catch'em all · Why Not to Trust Statistics ·
A critique of pure learning and what artificial neural networks can learn from animal brains ·
Visualization Types · Google Chart Gallery ·
Other Tools
|
See assignment 1 below.
|
EDA
|
| Wed May 12 |
Decision Trees
(UPDATED)
|
A Visual Introduction to Machine Learning ·
Decision Trees ·
Entropy
AI:AMA 18.2-3, ESL: 9.2, ML:APP 16.2
|
Big-O Notes
|
|
|
Fundamentals of Learning
(UPDATED)
|
7 Steps of Machine Learning ·
IID ·
Cross-validation ·
Bias-variance ·
No Free Lunch
AI: AMA 18.4-5, ESL 7.1-7.4, 7.10, ML:APP 1.4, 6.5
|
Course Notation Guide
|
|
| Fri May 14 |
Probabilistic Classifiers
(UPDATED)
|
Conditional probability (demo) ·
Naive Bayes ·
Probabilities and Battleship
ESL 4.3, ML: APP 2.2, 3.5, 4.1-4.2
|
Probability Notes
Probability Slides
|
|
|
Non-Parametric Models
(UPDATED)
|
K-nearest neighbours ·
Decision Theory for Darts ·
Norms
AI: AMA 18.8, ESL 13.3, ML:APP 1.4
|
|
|
| Mon May 17 |
Data Augmentation and Ensemble Methods
(UPDATED)
|
Ensemble Methods ·
Random Forests ·
Empirical Study ·
Kinect ·
Data Augmentation
AI: AMA 18.10, ESL: 7.11, 8.2, 15, 16.3, ML: APP 6.2.1, 16.2.5, 16.6
|
|
|
Clustering
(UPDATED)
|
Clustering ·
K-means clustering (demo) ·
K-Means++ (demo)
IDM 8.1-8.2, ESL: 14.3
|
|
Wed May 19
|
More Clustering
(UPDATED)
|
DBSCAN
(video
demo) ·
Hierarchical Clustering ·
Phylogenetic Trees
IDM 8.4
|
|
|
Outlier Detection
(UPDATED)
|
Empirical Study
IDM 8.3, ESL 14.3.12, ML:APP 25.5 |
|
| Fri May 21 |
Least Squares
(UPDATED)
|
Linear Regression ·
(demo,
2D data, 2D video) ·
Least Squares
Essence of Calculus ·
Partial Derivative ·
Gradient
ESL 3.1-2, ML:APP 7.1-3, AI:AMA 18.6
|
Calculus Notes
Linear Algebra Notes
Linear/Quadratic Gradients
|
|
Gradient Descent
(UPDATED)
|
Gradient Descent ·
|
|
| Wed May 26 |
Convex Functions and Robust Regression
(UPDATED)
|
Convex Functions ·
ML:APP 7.4
|
|
|
Bonus Lecture
(UPDATED)
|
MMD Chapter 3
|
|
Fri May 28
|
Feature Selection
(UPDATED)
|
Genome-Wide Association Studies ·
AIC ·
BIC
ESL 3.3, 7.5-7
|
|
|
Regularization
(UPDATED)
|
ESL 3.4., ML:APP 7.5, AI:AMA 18.4
|
|
Mon May 30
|
Gaussian RBF
(UPDATED)
|
RBF video ·
RBF and Regularization video
ESL 6.7, ML:APP 13.3-4
|
|
|
Linear Classifiers
(UPDATED)
|
Perceptron
ESL 4.5, ML:APP 8.5
Support Vector Machines
ESL 4.4, 12.1-2, ML:APP 8.1-3, 9.5 14.5, AI:AMA 18.9
|
|
Tue Jun 1
|
Midterm Exam
|
|
|
Wed Jun 2
|
Multi-Class Linear Classifiers
(UPDATED)
|
ML:APP 8.3.7 and 9.3-5, ESL 4.4
|
|
|
Feature Engineering
(UPDATED)
|
Gmail Priority Inbox
|
|
Fri Jun 4
|
Text and Image Data
(UPDATED)
|
Convolution GIF
Convolution Interactive Demo
ESL 12.3, ML:APP 14.1-4
|
|
|
Kernel Trick
(UPDATED)
|
ESL 12.3, ML:APP 14.1-4
|
|
Mon Jun 7
|
Stochastic Gradient
(UPDATED)
|
Stochastic Gradient
ML:APP 8.5
|
|
|
MLE and MAP
(UPDATED)
|
Maximum likelihood estimation
ML:APP 9.3-4
| Max and Argmax Notes |
Wed Jun 9
|
Principal Component Analysis
(UPDATED)
|
Principal Component Analysis
ESL 14.5, IDM B.1, ML:APP 12.2
|
|
|
More PCA
(UPDATED)
|
Making Sense of PCA
Singular Value Decomposition
Eigenface
| |
Fri Jun 11
|
Sparse Matrix Factorization
(UPDATED)
|
Non-Negative Matrix Factorization
NMF Paper
ESL 14.6, ML: APP 13.8
|
|
|
Recommender Systems
(UPDATED)
|
Recommender Systems
Netflix Prize
| |
Mon Jun 14
|
Multi-Dimensional Scaling
(UPDATED)
|
Nonlinear Dimensionality Reduction
t-SNE Demo
ESL 14.8-9, IDM B.2
|
|
|
Deep Learning
(UPDATED)
|
Google Video
What is a Neural Network?
Interactive Guide
ML:APP 16.5, ESL 11.1-4, AI: AMA 18.7
| |
| The FINAL will be from the material above. |
Wed Jun 16
|
More Deep Learning
(UPDATED)
|
Backpropagation
Fortune Article
Deep Learning References
Alchemy
ML:APP 28.3, ESL 11.5 |
|
|
Convolutional Neural Networks
(UPDATED)
|
Convolutional Neural Network
ML:APP 28.4, ESL 11.7
| |
The acroynms in the table above refer to the following textbooks:
- AI:AMA: Artificial Intelligence: A Modern Approach by Russell and Norvig
- ESL: The Elements of Statistical Learning by Hastie et al.
- ML:APP: Machine Learning: A Probabilistic Perspective by Kevin Murphy
- PRML: Pattern Recognition and Machine Learning by Christopher Bishop
- IDM: Introduction to Data Mining by Steinbach et al.
Related courses that have online notes
Homework Assignments
| Post Date |
Due Date |
Files |
Notes/Links |
| Mon May 10 |
Mon May 17
|
a1.pdf · a1.zip (contains the LaTeX template + code)
|
Setting up Python
|
| Sun May 16 |
Mon May 24
|
a2.pdf · a2.zip (contains the LaTeX template + code)
|
|
| Thu May 20 |
Fri May 28
|
a3.pdf · a3.zip (contains the LaTeX template + code)
|
|
| Thu May 27 |
Mon Jun 7
|
a4.pdf · a4.zip (contains the LaTeX template + code)
|
|
| Thu Jun 3 |
Fri Jun 11
|
a5.pdf · a5.zip (contains the LaTeX template + code)
|
|
| Sat Jun 12 |
Fri Jun 18
|
a6.pdf · a6.zip (contains the LaTeX template + code)
|
|
| Mon Jun 21 |
Wed Jun 30
|
a7.pdf · a7.zip (contains the LaTeX template + code)
|
|
- Submission will happen through Gradescope.
- Instructions Comprehensive submission instructions will be made available soon.
- Handwritten submissions are not allowed. You will need to use LaTeX.
- You must work alone for Assignment 1. For Assignment 2 onwards, you may work with a partner or work alone. You can make this decision independently for each assignment. Make your partnership details explicit in your report.
- Suggestion: if you have never used LaTeX and/or Python before, try to find a partner who has, especially for the early assignments when these tools are new to you.
- Late submissions are not allowed.
- Assignment updates It happens sometimes that the instructor needs to make changes to an assignment after it is posted. If this happens we will notify you on piazza and you can download a new version of the assignment as required.
- Citation/Acknowledgement If you use information from students outside your group or from online sources including code snippets from Stack Overflow, cite this at the start of each question. You will receive a mark of 0 for the assignment (and possibly other consequences) if you are found copying from other sources without citation.
- Regrading policy If you perceive a problem with your homework grade, you have 1 week to raise a concern, from the time that your grades were posted. After that, your grade is final.
- See lecture 1 for further info.
Previous Offerings
- We will be going through the same path as the previous offerings of the course. You may find the following links useful.
- Mike's CPSC 340 videos on YouTube and slides, notebooks, and readings here.
- Mark's CPSC 340 material here.
