CS446/ECE449: Machine Learning (Spring 2020)

Course Information

The goal of Machine Learning is to build computer systems that can adapt and learn from data. In this course we will cover three main areas, (1) discriminative models, (2) generative models, and (3) reinforcement learning models. In particular we will cover the following: linear regression, logistic regression, support vector machines, deep nets, structured methods, learning theory, kMeans, Gaussian mixtures, expectation maximization, VAEs, GANs, Markov decision processes, Q-learning and Reinforce.

Pre-requisites: Probability, linear algebra, and proficiency in Python.

Recommended Text: (1) Machine Learning: A Probabilistic Perspective by Kevin Murphy, (2) Deep Learning by Ian Goodfellow and Yoshua Bengio and Aaron Courville, (3) Pattern Recognition and Machine Learning by Christopher Bishop, (4) Graphical Models by Nir Friedman and Daphne Koller, and (5) Reinforcement Learning by Richard Sutton and Andrew Barto.


3 credit: Homework 33%, Midterm 33%, Final 33%

4 credit: Homework 16.6%, Scribe 16.6%, Midterm 33%, Final 33%

The lowest homework grade will be dropped (the scribe cannot be dropped) and we will compute the average score of the remaining 9 assignments.
Grading policy is subject to change.

TA Hours:
Time: Tuesdays/Thursdays: 5-6:30pm (2 TAs) on days before homework deadlines and 5-6pm (1 TA) otherwise.
Room: ECEB 2015.
No TA hours on February 27 and April 9.

Late Policy: No late submission will be accepted after the due date.

Midterm: March 12, 12:30pm - 13:45. Room: TBD.
Final Exam: May 14, 7-10pm. Rooms: 1002 ECEB, 1013 ECEB, 3013 ECEB, 3017 ECEB.

Instructor & TAs

Alexander Schwing

Email: aschwing[at]illinois.edu
Office Hour: TBD
Room: CSL 103
Website: [link]

Jyoti Aneja

Teaching Assistant
Email: janeja2[at]illinois.edu
Website: [link]

Safa Messaoud

Teaching Assistant
Email: messaou2[at]illinois.edu
Website: [link]

Junrui Ni

Teaching Assistant
Email: junruin2[at]illinois.edu
Website: [link]

Jason Zhongzheng Ren

Teaching Assistant
Email: zr5[at]illinois.edu
Website: [link]

Amr Martini

Teaching Assistant
Email: ammartn3[at]illinois.edu
Website: [link]

Class Time & Location

Class Time: Tuesday, Thursday 12:30-1:45PM
Location: ECEB 1002 (map)

Course Discussions

Piazza for discussions: [link]
GradeScope for assignments (self-enrollment code 9ZG73B): [link]

Homework & Scribe

Homework: [link]
Scribe: [link]


The syllabus is subject to change.

Lecture 1 Jan. 21 Introduction (Nearest Neighbor) [Link] [Link2]    
Lecture 2 Jan. 23 Linear Regression [Link] [Link2]    
Lecture 3 Jan. 28 Logistic Regression [Link] [Link2]    
Lecture 4 Jan. 30 Optimization Primal [Link] [Link2]    
Lecture 5 Feb. 4 Optimization Dual [Link] [Link2]    
Assignment Due Feb. 6 Assignment 1 Due (11:59AM Central Time)
Lecture 6 Feb. 6 Support Vector Machine [Link] [Link2]    
Lecture 7 Feb. 11 Multiclass Classification and Kernel Methods [Link] [Link2]    
Assignment Due Feb. 13 Assignment 2 Due (11:59AM Central Time)
Lecture 8 Feb. 13 Deep Nets 1 (Layers) [Link] [Link2]    
Lecture 9 Feb. 18 Deep Nets 2 (Backpropagation + PyTorch) [Link] [Link2]    
Assignment Due Feb. 20 Assignment 3 Due (11:59AM Central Time)
Lecture 10 Feb. 20 PyTorch [Link] [Link2]    
Lecture 11 Feb. 25 Ensemble Methods (Boosting/Random Forest/Deep Nets) & Regularization/Cross-Val [Link] [Link2]    
Assignment Due Feb. 27 Assignment 4 Due (11:59AM Central Time)
Lecture 12 Feb. 27 Structured Prediction (exhaustive search, dynamic programming) [Link] [Link2]    
Lecture 13 Mar. 3 Structured Prediction (ILP, LP relaxation, message passing, graph cut) [Link] [Link2]    
Assignment Due Mar. 5 Assignment 5 Due (11:59AM Central Time)
Lecture 14 Mar. 5 Learning in Structured Models [Link] [Link2]    
Lecture 15 Mar. 10 Review [Link] [Link2]    
Assignment Due Mar. 12 Assignment 6 Due (11:59AM Central Time)
Lecture 16 Mar. 12 Midterm [Link]    
Lecture 17 Mar. 17 Spring Break [Link] [Link2]    
Lecture 18 Mar. 19 Spring Break [Link] [Link2]    
Lecture 19 Mar. 24 Learning Theory [Link] [Link2] [PreRec] [Rec]
Lecture 20 Mar. 26 PCA, SVD [Link] [Link2] [PreRec] [Rec]
Lecture 21 Mar. 31 k-Means [Link] [Link2]   [Rec]
Lecture 22 Apr. 2 Gaussian Mixture Models [Link] [Link2]   [Rec]
Assignment Due Apr. 7 Assignment 7 Due (11:59AM Central Time)
Lecture 23 Apr. 7 Expectation Maximization [Link] [Link2]   [Rec]
Lecture 24 Apr. 9 Hidden Markov Models [Link] [Link2]   [Rec]
Lecture 25 Apr. 14 Variational Auto-Encoders [Link] [Link2] [PreRec] [Rec]
Lecture 26 Apr. 16 Generative Adversarial Nets [Link] [Link2] [PreRec] [Rec]
Assignment Due Apr. 21 Assignment 8 Due (11:59AM Central Time)
Lecture 27 Apr. 21 Autoregressive Methods [Link] [Link2] [PreRec] [Rec]
Lecture 28 Apr. 23 MDP [Link] [Link2] [PreRec] [Rec]
Assignment Due Apr. 28 Assignment 9 Due (11:59AM Central Time)
Lecture 29 Apr. 28 Q-Learning [Link] [Link2] [PreRec] [Rec]
Lecture 30 Apr. 30 Policy Gradient, Actor-Critic [Link] [Link2]   [Rec]
Assignment Due May. 5 Assignment 10 Due (11:59AM Central Time)
Lecture 31 May. 5 Review [Link] [Link2]   [Rec]
Exam TBD Final Exam