Applied Machine Learning CS-498 Spring 2019

About

When

3:30 PM - 4:45 PM Tuesday/Thursday

Location

1320 Digital Computer Library

Professor

Trevor Walker - taw@illinois.edu

TAs

Section 498-AML

  • Chase Duncan
  • Krishna Kothapalli
  • Ankit Kumar
  • Peiye Zhuang

Section 498-AMO

 

  • Krishna Dusad
  • Tiantian Fang
  • Ashwin Ramesh
  • Renxuan Wang
  • Ching-Hua Yu
  • Xiaoming Zhao

Syllabus

We will start at the beginning of the textbook and proceed to the end, covering approximately one chapter per week. There are 14 substantive chapters and 15 weeks; this is to allow a little spreading out.

Read the textbook. It was written specifically for this course, AND it's free. We will split time in lecture between sketching important points described in the text and solving problems. If you haven't read the text, this might be quite puzzling.

Topics

  • Classification
    • Learning To Classify
    • SVM’s and Random Forests
  • High Dimensional Data
    • Principal Component Analysis
    • Low Rank Approximations
    • Canonical Correlation Analysis
  • Clustering
    • Agglomerative and Divisive Clustering
    • K-Means
    • Mixture Models and EM Algorithm
  • Regression
    • Linear Regression and Least Squares
    • Choosing and Managing Models
    • Kernels and Neighbors
  • Graphical Models
    • Hidden Markov Models
    • Conditional Random Field Models for Sequences
    • Mean Field Inference
  • Deep Networks
    • Simple Neural Networks
    • Image Classification with Neural Networks
    • Autoencoding with Neural Networks
    • Latent Variable Models
  • Boosting
  • A Little Learning Theory

Required Text

Important: This will change as we purge typos. If you print it, you'll have to do it again. We recommend working from the electronic copy if possible.

Applied Machine Learning, D.A. Forsyth, (approximate 18'th draft)

Office Hours

Section 498-AML

Location: Siebel 0207

  • Chase Duncan - Mon 10:00-11:00am, Thu 2:15-3:15pm
  • Krishna Kothapalli - Tue 10:00-11:00am, Thu 10:00-11:00am
  • Ankit Kumar - Mon 10:30am-12:30pm
  • Peiye Zhuang - Tue 2:00-3:00pm, Wed 2:00-3:00pm

Section 498-AMO

See Coursera Site for TA hours and instructions

Homeworks

There is a total of 10 homeworks. There will be no final exam - one homework will be designated a take-home final.

Tentative Homework Schedule

Dates and topics may be adjusted later. Tentative dates and topics are shown in italics.

# Due Date Topic Link
1 Mon 1/28/19, 11:59 PM CST (AML)
Thur 1/31/19, 11:59 PM CST (AMO)*
Classification w/Naive Bayes Homework 1
2 Mon 2/4/19, 11:59 PM CST
Wed 2/6/19, 11:59 PM CST
Classification w/SVM Homework 2 (Important update on 2/2)
3 Mon 2/11/19, 11:59 PM CST Smoothing Noise w/PCA Homework 3
4 Mon 2/18/19, 11:59 PM CST More PCA Homework 4
5 Mon 3/4/19, 11:59 PM CST Classifying w/Variable Length Inputs Homework 5
6 Mon 3/11/19, 11:59 PM CST Outlier Detection Homework 6
7 Mon 4/1/19, 11:59 PM CST
(updated 3/14)
Text bag-of-words Search and classification
(updated 3/14)
Homework 7
8 Mon 4/15/19, 11:59 PM CST
(updated 4/1)
Classification with Artificial Neural Networks Homework 8
9 Wed 4/24/19, 11:59 PM CST Variational Autoencoders Homework 9
10 Wed 5/1/19, 11:59 PM CST
(updated 4/19)
Deep Neural Networks (Take-home Final)
(updated 4/19)
Homework 10

* AMO students are allowed a 3-day extension for homework 1 due to the late release of Coursera materials.

Submission Policy

We will use Gradescope to submit homework and provide grading feedback. Additional Gradescope details will be provided once we are closer to the homework deadlines.

Late homeworks will be assessed a 5% penalty per day, up to a maximum of 30%. After the homework due date is passed, the most recent homework submission will be graded and the relevant penalty will be applied.

After grading is complete, one resubmission. The resubmission will be regraded, with with a 30% late penalty, and the score will be updated. Please submit a regrade request through Gradescope if you resubmit after grading. Five days after grading is complete, homework submission will be locked and homeworks cannot be submitted or resubmitted without obtaining an explicit exception.

No resubmissions of regular homeworks will be accepted after May 1st (last day of instruction). The Take-home final may not be submitted after May 10th (last date of final examination period).

Late Extensions

Very few exceptions will be allowed, excepting serious illness or family emergencies. Please submit a written request (i.e. email) to obtain exceptions.

Students who add the class late with have up to 5 days to complete homework 1 after their add date without late penalty for that homework. Students who add late must request the extension via email (mostly for bookkeeping purposes). Other homework will not be allowed exceptions due to late adds.

Programming Languages

Python is the recommended language for homework, with the exception of homework 7, which must be done in R. Also, Homework 11 must be done in Python. Students may use other languages if desired for homeworks other than 7 & 11.

Students using Python may consider using a free online notebook such as Google CoLab or Microsoft Azure. These provide free notebook and computation with convenient web access.

Collaboration

All homeworks can be done in teams of up to 2 people. The teams may consist of members from any section of the course (online, on-campus, etc.). Both group members should contribute to the work equally. Each student must submit their homework individually, with team members explicitly specified in homework submission.

Citation policy

In this course, students are allowed to use library code to solve high-level problems, unless an assignment specifically requests that students implement an algorithm completely by themselves. Students should always cite any library code they have used. Students must cite online code references (documentation examples, StackOverflow, Piazza posts, Slack discussions, etc.) that they have referred to. If students collaborate to any extent, they must cite each other (name and NetID where appropriate) in their code comments and PDF reports. Students may talk to members of other groups and cite them, but they are limited to at most two students in a submission group.)

In summary, this policy reflects the University policy on plagiarism as it applies to academic writing.

Students must cite all references, including any code they have used that they did not write themselves. Failure to cite references will be considered an academic integrity violation and be pursued according to University policy, which may include receiving a failing grade on an assignment or in the entire course.

Citations do not need to follow any specific format (such as ACM style, etc.) but should mention the author's name and where the cited work can be found (including a URL, if applicable). In code, a citation can be left in a comment. The PDF report should summarize any citations.

Grading Formula

(Updated) The policy on homeworks has not changed. There will only be 10 homeworks, and you will still be able to drop your lowest one  excluding HW1 and HW10. 

All homeworks will receive equal weight in the determining the final grade. In determining the final grade, the lowest scored homework between homework 2 to 9 will be dropped, and the remaining 9 homework scores will be averaged. Homework 1 and 10 will always be counted in the final grade (updated)

Once the final scores are determined, a grading scale will be determined to assign a letter grade. Historically, the score distribution has been bimodal so we won't attempt to fit a bell-curve.

Piazza

Piazza - Spring 2019 AML 498

Current Movies

Data Hi-res Low-res Topic Additional Materials
15-Jan Video Video Class Overview, Classification  
17-Jan Video Video Classification, Evaluation, Cross-Validation, Nearst Neighbors, Naive Bayes simple_nb.py
22-Jan Video Video Cross-Validation, Naive Bayes, Decision Trees  
24-Jan Video Video Decision Trees, Support Vector Machines  
29-Jan Video Video Support Vector Machines, High-Dimensional Data sgd_demo.ipynb
(In colab format)
31-Jan Video Video SVM Clarifications, High-Dimensional Data, Principal Component Analysis  
5-Feb Video Video Principal Component Analysis, NIPALS  
7-Feb Video Video Principal Component Analysis via SVD, Multi-Dimensional Scaling, Canonical Corrolation Analysis  
12-Feb Video Video Homework 2 Debugging, Homework 3 Demo, K-Means Clustering  
14-Feb Video Video Vector Quantization, Vector Quantization Demo, Expectation-Maximization  
19-Feb Video Video Gaussian Mixture Models & Expectation-Maximization, Text Processing with Bag-of-words  
21-Feb Video Video Text Latent Sematic Analysis and Topic Models, Linear Regression  
26-Feb Video Video Topic Modelling Demo, Linear Regression, Leverage, Cook's Distance  
28-Feb Video Video Spotting Outliers: Leverage, Cook's Distance, Standardized Residuals, Bias-Variance Tradeoff  
5-Mar Video Video Robust Regression, Generalized Linear Models, Logistic Regression, Deviance, Classifier Thresholds & Score Distributions, ROC & PR Curves  
7-Mar Video Video L1 Regularization, Kernel Regression, Markov Chains  
12-Mar Video Video Hidden Markov Models Simple HMM Speech Recognition
14-Mar Video Video Hidden Markov Models, Intro to Conditional Random Fields.
Due to glitch, the videos were recorded incorrectly. Short Rerecorded video available here.
 
26-Mar Video Video HMM Review, Graphical Models, Variational Inference  
28-Mar Video Video Variational Inference, Intro to Neural Networks  
2-Apr Video Video Neural Networks  
4-Apr Video Video Convolutional Neural Networks  
9-Apr Video Video Neural Networks  
11-Apr Video Video Recurrent Neural Networks  
16-Apr Video Video Neural Networks, Autoencoders  
18-Apr Video Video Boosting  
23-Apr Video Video Boosting, Reinforcement Learning  
25-Apr Video Video Reinforcement Learning  
30-Apr Video Video Recommendation Systems  

Previous Movies

Fall 2018 Movies

Date AM-low-res PM-low-res AM-hi-res PM-hi -res
28-Aug        
30-Aug 30 Aug 30 Aug 30 Aug 30 Aug
4-Sep 4 Sep 4 Sep 4 Sep 4 Sep
6-Sep 6 Sep 6 Sep 6 Sep 6 Sep
11-Sep 11 Sep 11 Sep 11 Sep 11 Sep
13-Sep 13 Sep 13 Sep 13 Sep 13 Sep
18-Sep 18 Sep 18 Sep 18 Sep 18 Sep
20-Sep 18 Sep 20 Sep 18 Sep 20 Sep
25-Sep Travel 25 Sep Travel 25 Sep
27-Sep Travel 27 Sep Travel 27 Sep
2-Oct Travel 2 Oct Travel 2 Oct
4-Oct 4 Oct 4 Oct 4 Oct 4 Oct
9-Oct 9 Oct 9 Oct 9 Oct 9 Oct
11-Oct 11 Oct 11 Oct 11 Oct 11 Oct
16-Oct 16 Oct 16 Oct 16 Oct 16 Oct
18-Oct 18 Oct 18 Oct 18 Oct 18 Oct
23-Oct 23 Oct 23 Oct 23 Oct 23 Oct
25-Oct 25 Oct 25 Oct 25 Oct 25 Oct
30-Oct 30 Oct 30 Oct 30 Oct 30 Oct
1-Nov 1 Nov 1 Nov 1 Nov 1 Nov
6-Nov Travel 6 Nov Travel 6 Nov
8-Nov Travel 8 Nov Travel 8 Nov
13-Nov Travel 13 Nov Travel 13 Nov
15-Nov 15 Nov 15 Nov 15 Nov 15 Nov
27-Nov 27 Nov 27 Nov 27 Nov 27 Nov
1-Dec 1 Dec 1 Dec 1 Dec 1 Dec
4-Dec 4 Dec Use am movie 4 Dec Use am movie
6-Dec 6 Dec Use am movie 6 Dec Use am movie
1/21/19