ECE544: Pattern Recognition (Fall 2020)

Course Information

The goal of Pattern Recognition is to find structure in data. In this course we will cover three main areas, (1) discriminative models, (2) generative models, and (3) reinforcement learning models. In particular we will cover the following: linear regression, logistic regression, support vector machines, deep nets, structured methods, learning theory, kMeans, Gaussian mixtures, expectation maximization, VAEs, GANs, Markov decision processes, Q-learning and Reinforce.

Pre-requisites: Probability, linear algebra, and proficiency in Python.

Recommended Text: (1) Machine Learning: A Probabilistic Perspective by Kevin Murphy, (2) Deep Learning by Ian Goodfellow and Yoshua Bengio and Aaron Courville, (3) Pattern Recognition and Machine Learning by Christopher Bishop, (4) Graphical Models by Nir Friedman and Daphne Koller, and (5) Reinforcement Learning by Richard Sutton and Andrew Barto.

Course Deliverables:
(1) One course presentation (see Courseplan Lecture Group ID column for your team members)
(2) One project (see Courseplan Project Team ID column for your team members)

Grading:

1/3 class presentation, 1/3 project, 1/3 final; + 1/3 bonus for really impressive projects


Grading policy is subject to change.

TA Hours:
Time: Friday at 5pm.

Final Exam: December 8, 11am-12:30pm.

Instructor & TAs

Alexander Schwing

Instructor
Email: aschwing[at]illinois.edu
Office Hour: Friday at 8am
Website: [link]

Safa Messaoud

Teaching Assistant
Email: messaou2[at]illinois.edu
Website: [link]

Jason Zhongzheng Ren

Teaching Assistant
Email: zr5[at]illinois.edu
Website: [link]

Class Time & Location

Class Time: Tuesday, Thursday 11:00AM-12:15PM

Course Discussions

Piazza for discussions: [link]

Practice Material

Practice Material: [link]



Lectures

The syllabus is subject to change.

EventDateDescriptionMaterialsPre-RecordingRecording
Lecture 1 Aug. 25 Introduction (Nearest Neighbor) [Slides] [Slides Split] [PreRec] [Rec]
Lecture 2 Aug. 27 Linear Regression [Slides] [Slides Split] [PreRec] [Rec]
Lecture 3 Sep. 1 Logistic Regression [Slides] [Slides Split] [PreRec] [Rec]
Lecture 4 Sep. 3 Optimization Primal [Slides] [Slides Split] [PreRec] [Rec]
Lecture 5 Sep. 8 Optimization Dual [Slides] [Slides Split] [Team Slides] [PreRec] [Rec]
Lecture 6 Sep. 10 Support Vector Machine [Slides] [Slides Split] [Team Slides] [PreRec] [Rec]
Lecture 7 Sep. 15 Multiclass Classification and Kernel Methods [Slides] [Slides Split] [Team Slides] [PreRec] [Rec]
Lecture 8 Sep. 17 Deep Nets 1 (Layers) [Slides] [Slides Split] [Team Slides] [PreRec] [Rec]
Lecture 9 Sep. 22 Deep Nets 2 (Backpropagation + PyTorch) [Slides] [Slides Split] [Team Slides] [PreRec] [Rec]
Lecture 10 Sep. 24 Ensemble Methods (Boosting/Random Forest/Deep Nets) & Regularization/Cross-Val [Slides] [Slides Split] [Team Slides] [PreRec] [Rec]
Lecture 11 Sep. 29 Structured Prediction (exhaustive search, dynamic programming) [Slides] [Slides Split] [Team Slides] [PreRec] [Rec]
Lecture 12 Oct. 1 Learning Theory [Slides] [Slides Split] [Team Slides] [PreRec] [Rec]
Lecture 13 Oct. 6 Review [Slides] [Slides Split]   [Rec]
Lecture 14 Oct. 8 PCA, SVD [Slides] [Slides Split] [Team Slides] [PreRec] [Rec]
Lecture 15 Oct. 13 k-Means [Slides] [Slides Split] [Team Slides] [PreRec] [Rec]
Lecture 16 Oct. 15 Gaussian Mixture Models [Slides] [Slides Split] [Team Slides] [PreRec] [Rec]
Lecture 17 Oct. 20 Expectation Maximization [Slides] [Slides Split] [Team Slides] [PreRec] [Rec]
Lecture 18 Oct. 22 Hidden Markov Models [Slides] [Slides Split] [Team Slides] [PreRec] [Rec]
Lecture 19 Oct. 27 Variational Auto-Encoders [Slides] [Slides Split] [Team Slides] [PreRec] [Rec]
Lecture 20 Oct. 29 Generative Adversarial Nets [Slides] [Slides Split] [Team Slides] [PreRec] [Rec]
Election Nov. 3 Election Day      
Lecture 21 Nov. 5 Review [Slides] [Slides Split]   [Rec]
Lecture 22 Nov. 10 Autoregressive Methods [Slides] [Slides Split] [Team Slides] [PreRec] [Rec]
Lecture 23 Nov. 12 MDP [Slides] [Slides Split] [Team Slides] [PreRec] [Rec]
Lecture 24 Nov. 17 Q-Learning [Slides] [Slides Split] [Team Slides] [PreRec] [Rec]
Lecture 25 Nov. 19 Review/Policy Gradient, Actor-Critic [Slides] [Slides Split] [PreRec] [Rec]
Fall Break Nov. 24 Fall Break      
Fall Break Nov. 26 Fall Break      
Project Dec. 1 Project Presentations     [Rec]
Project Dec. 3 Project Presentations     [Rec]
Exam Dec. 8 Final Exam