Schedule

Optional readings from the textbooks are listed below for each module. Readings from ISLR are recommended, while those from ESL (in parentheses) are optional and supplemental.

All lecture notes as .qmd files are available here. R code for all lectures is available here.

1 The Learning Procedure - Models, Fitting, Model Selection

Topics: Learning through statistician and algorithmic lenses, model selection; cross validation

Learning Objectives:

1. Formulate learning problems in terms of statistical models, estimators, and model selection
2. Identify criteria for good statistical models, estimators, and model selection metrics

Handouts and Resources:

• Programming in R .Rmd, .pdf
• Using RMarkdown .Rmd, .pdf
• Overview of R and Tidyverse (Slides from last term)
• Overview of git and version control (Slides from last term)

Date	Topic	Readings	Deadlines
Sep 8	(no class, Imagine UBC)
Sep 10	Class Overview (slides) Probability and SVD Review (notes)
Sep 15	Introduction to Learning, Regression (notes)	ISLR 2.1 (ESL 2.4, 2.6)
Sep 17	Learning (cont.), Classification (notes)	ISLR 4.3 (ESL 4.4)
			Lab 00 (Sep 14-19) Download lab00.Rmd
Sep 22	Model Selection, Cross Validation (notes)	ISLR 5.1 (ESL 2.9, 7.10)

2 Bias-Variance Tradeoff, Linear Methods

Topics: bias/variance tradeoff; regularized regression (ridge and lasso); non-linearities via basis functions; advanced model selection and analysis

Learning Objectives:

1. Decompose prediction error into bias and variance components
2. Implement regularized versions of linear regression (ridge, lasso) and understand their impact on bias and variance
3. Implement basis expansions for linear regression and understand their impact on bias and variance
4. Apply closed-form selection techniques to linear methods, and identify factors in the formula that affect bias and variance

Date	Topic	Readings	Deadlines
Sep 24	Bias-Variance Tradeoff (notes)	ISLR 2.2 (ESL 7.1-7.3)	HW 1 due Sep 26 Download hw01.Rmd Instructions preamble.tex
			Lab 01 (Sep 21-26)
Sep 29	Ridge Regression (notes)	ISLR 6.2.1 (ESL 3.4.0-3.4.1)
Oc1 01	Lasso Regression (notes)	ISLR 6.2.2-6.2.3 (ESL 3.4.2-3.4.3)
			Lab 02 (Sep 28-Oct 03)
Oct 06	Basis Functions (notes)	ISLR 7.1, 7.4 (ESL 5.1-5.3)
			Lab 03 (Oct 05-10)

3 Nonparametric Methods, Curse of Dimensionality

Topics: kNN; trees; kernel machines; curse of dimensionality

Learning Objectives:

1. Analyze how dimensionality affects the performance of parametric vs nonparametric methods
2. Implement nonparametric methods (kNN, kernel smoothing, kernel machines) and analyze their properties
3. Write the parametric version of nonparametric methods (e.g. kernel ridge regression) and vice versa

Date	Topic	Readings	Deadlines
Oct 08	Model Selection for Linear Methods (notes)	(ESL 7.6-7.7)	HW 2 due Oct 10
Oct 13	Kernel Machines (notes)	ISLR 3.5 (ESL 2.3.2, 5.4.1)
Oct 15	kNN, Curse of Dimensionality (notes)	ISLR 3.5, 8.1 (ESL 2.3.2, 5.4.1, 9.2)
			Lab 04 (Oct 12-17)
Oct 20	Catch-up if needed

Midterm Exam

Date	Topic
Oct 27	MIDTERM EXAM (In Class)

• In person attendance is required (per Faculty of Science guidelines)
• You must bring your computer as the exam will be given through Canvas
• Please arrange to borrow one from the library if you do not have your own. Let me know ASAP if this may pose a problem.
• You may bring 2 sheets of front/back 8.5 × 11 inch paper with handwritten notes you want to use. No other materials will be allowed.
• There will be no required coding, but I may show code or output and ask questions about it.
• It will be entirely multiple choice / True-False / matching, etc. Delivered on Canvas.

4 Unsupervised Learning

Topics: dimension reduction and clustering

Learning Objectives:

1. Implement dimensionality reduction techniques (PCA, kernel PCA) and analyze their impact on data representation
2. Apply clustering algorithms (k-means, Gaussian mixture models) and evaluate their performance using appropriate metrics

Date	Topic	Readings	Deadlines
Oct 22	Trees (catch up from last module) (Notes)	ISLR 8.1, 4.2.0 (ESL 9.2)
Oct 29	Dimensionality Reduction, Intro to Unsupervised Learning (Notes)	ISLR 12.2 (ESL 14.5.1, 14.5.4)
			Lab 05 (Oct 26-31)
Nov 03	Clustering (Notes)	ISLR 12.4.1 (ESL 14.3)

5 Ensembles, Black-Box Methods

Topics: ensembles; bootstrap; bagging; boosting; random forests

Learning Objectives:

1. Implement bootstrap and ensembling methods, reason through computational tradeoffs
2. Differentiate ensemble methods that reduce bias or variance
3. Utilize “hidden advantages” of ensembles around feature importance, uncertainty quantification, etc.
4. Identify assumptions in black-box methods of uncertainty quantification, variance reduction, and bias reduction

Date	Topic	Readings	Deadlines
Nov 05	The Bootstrap (Notes)	ISLR 5.2 (ESL 7.11, 8.2)	HW 3 due Nov 07
			Lab 07 (Nov 02-07) NOTE: SKIP LAB 06
Nov 10	(no class, Midterm Break)
Nov 12	catch up if needed
Nov 17	Bagging, Random Forests (Notes)	ISLR 8.2.0-8.2.2 (ESL 8.7, 15.1-15.3)
Nov 19	Boosting (Notes)	ISLR 8.2.3 (ESL 10.1-10.5, 10.9)
			Lab 08 (16-21)

6 Deep Learning

Topics: neural networks; deep learning architectures; generative AI

Learning Objectives:

1. Construct a basic neural network architecture from simple mathematical building blocks
2. Articulate the effects of depth and width on the representational capacity and generalization of neural networks
3. Connect neural networks to other methods covered in the course (basis functions, kernel methods, boosting methods)
4. Derive the backpropagation algorithm
5. Evaluate modern neural network architectures for different problem types

Date	Topic	Readings	Deadlines
Nov 24	Introduction to Neural Networks (Notes)	ISLR 10.1-10.2 (ESL 11.1, 11.3)
Nov 26	Neural Network Optimization (Notes)	ISLR 10.7-10.8 (ESL 11.4)	HW 4 due Nov 28
			Lab 09 (Nov 23-28)
Dec 01	Generalization (Notes)
Dec 02	Catch-up if needed & Review

Final Exam

Important

Do not make any plans to leave Vancouver before the final exam date is announced.

• In person attendance is required (per Faculty of Science guidelines)
• You may bring 2 sheets of front/back 8.5 × 11 inch paper with handwritten notes you want to use. No other materials will be allowed.