CS 307: Modeling and Learning in Data Science

Course Information

• Instructor: Mahesh Viswanathan
• Lectures: M/W 9:30am to 10:45am in Siebel 1302.
• Discussion Lab: F 9:30am to 10:45am in Siebel 1302.
• Office Hours: After class Monday, Wednesday, Friday until 11:15, and by appointment.
• Discussion Group: Campuswire
  Discussion board link and access code can be found here
• PrairieLearn site for the class can be found here

Resources

• Mediaspace channel with lecture recordings.
• Understanding Machine Learning: From Theory to Algorithms by Shai Shalev-Shwartz and Shai Ben-David.
• Stanford CS 229 Notes by Andrew Ng and updated by Tengyu Ma.
• Probabilistic Machine Learning: An Introduction by Kevin Murphy.
• Linear Algebra Review by Zico Kolter and updated by Chuong Do.

Course Overview. Introduction to the use of classical approaches in data modeling and machine learning in the context of solving data-centric problems. A broad coverage of fundamental models is presented, including linear models, unsupervised learning, supervised learning, and deep learning. A significant emphasis is placed on the application of models in Python and the interpretability of the results. This is one of the core classes in the X+DS programs.

Prerequisites. Calculus (MATH 220 or MATH 221); Python programming especially familiarity with NumPy, Pandas, and Mathplotlib (STAT 207); Probability and Statistics (STAT 207); Linear Algebra (one of MATH 225, MATH 257, MATH 415, MATH 416, ASRM 406).

Graded Work. Your graded work will consist of discussion lab attendance (4.5% total), homework/MPs (20.5% total), midterms (50% total), and final exam (25% total).

• Discussion Labs: Held every week (unless there is a midterm), for a total of 12 labs during the entire semester. Attending a lab gets you 0.5% of the total grade. You can get a maximum of 4.5% of the total grade through discussion lab attendance. For example, if you attend 4 labs during the entire semester, you will get 2%, 7 labs 3.5%, and attending 9 or more labs will get you the full 4.5% possible for this category.
• Homework/MPs: Every week a homework/MP will be assigned during the lab session. These are to be solved on PrairieLearn. The homework will be started during the discussion lab, and must be turned by the following Wednesday (midnight). There will be a total of 12 homework/MPs during the semester (one for each discussion lab). We will drop your 3 lowest scores when calculating your total grade for homework/MPs. In other words, your top 9 scores will be used to calculate your homework/MP grade.
• Midterms: There will be 3 midterms during the semester held either during a discussion lab or lecture time (see dates below). They will test competence on material covered since the previous midterm. To calculate your midterm total, your top 2 scores will be used. In other words, your lowest midterm score will be dropped.
• Final: There will be one comprehensive final. Based on the campus schedule for final exams, the date for the final is December 12.

Syllabus Statements. A link to syllabus statements is here.

Acknowledgements. Course contents inspired by conversations with and materials from David Forsyth, Bo Li, Hongye Liu, Marco Morales, Ehsan Saleh, and Matus Telgarsky. Website design is due to Manoj Prabhakaran.

Lecture videos. Lecture schedule below is tentative, but exam dates are fixed and will not change. Mediaspace channel with video recordings is here.

• Lecture 01 [Aug 22]: Course Introduction. [video]
• Lecture 02 [Aug 24]: Introduction to Supervised Learning and Linear Regression. Chapter 1 (until Section 1.1) of [Ng-Ma] and Section 4.1 of [LinAlg]. [video, scribbles]
• Lab 01 [Aug 26]: Python basics
• Lecture 03 [Aug 29]: Linear Regression: Normal Equations. Section 1.2 of [Ng-Ma]. [video, scribbles]
• Lecture 04 [Aug 31]: Linear Regression: Gradient descent and Stochastic gradient descent. Linear Classification: Halfspaces. Section 1.1 and 2.2 of [Ng-Ma]. [video, scribbles]
• Lab 02 [Sep 02]: Linear Regression. (10th day)
• [Sep 05]: Labor Day. No class.
• Lecture 05 [Sep 07]: Linear Classification: Perceptron. Section 2.2 of [Ng-Ma]. [video, scribbles]
• Lab 03 [Sep 09]: Classification
• Lecture 06 [Sep 12]: Logistic Regression. Chapter 2 of [Ng-Ma]. [video, scribbles]
• Lecture 07 [Sep 14]: Generalized Linear Models. Chapter 3 of [Ng-Ma]. [video, scribbles]
• Lab 04 [Sep 16]: Logistic Regression
• Lecture 08 [Sep 19]: Decision Trees. Section 18.2 of [Shai²]. [video, scribbles]
• Lecture 09 [Sep 21]: Nearest Neighbors, Decision Trees and Random Forests. Chapters 18 (not including 18.1) and 19.1 of [Shai²]. [video, scribbles]
• Lab 05 [Sep 23]: Nearest Neighbors and Decision Trees
• Lecture 10 [Sep 26]: Encoding data as features. Section 1.5 of [Murphy]. Lecture on zoom. [video, scribbles]
• Lecture 11 [Sep 28]: Midterm Review. Lecture on zoom. [video, scribbles]
• Lab [Sep 30]: Midterm 1
• Lecture 12 [Oct 03]: Convex Optimization. Section 12.1 of [Shai²]. [video, scribbles]
• Lecture 13 [Oct 05]: Support Vector Machines. Sections 6.1 to 6.4 and 6.7 of [Ng-Ma]. [video, scribbles]
• Lab 06 [Oct 07]: SVMs.
• Lecture 14 [Oct 10]: Dual SVM. Sections 6.5 and 6.6 of [Ng-Ma]. [video, scribbles]
• Lecture 15 [Oct 12]: Kernel Methods. Chapter 5 of [Ng-Ma]. [video, scribbles]
• Lab 07 [Oct 14]: Kernel Methods
• [Oct 14]: Drop deadline
• Lecture 16 [Oct 17]: Generative Models. Chapter 4 (up to section 4.2) of [Ng-Ma]. [video, scribbles]
• Lecture 17 [Oct 19]: Naive Bayes, Bias and Variance. Section 4.2 and 8.1 of [Ng-Ma]. Lecture on zoom. [video, scribbles]
• Lab 08 [Oct 21]: Generative Models.
• Lecture 18 [Oct 24]: Regularization and Model Selection. Chapter 9 until end of section 9.3 of [Ng-Ma]. Lecture on zoom. [video, scribbles]
• Lecture 19 [Oct 26]: Midterm Review. Lecture on zoom. [video]
• Lab [Oct 28]: Midterm 2
• Lecture 20 [Oct 31]: Introduction to Neural Nets. Chapter 7 upto section 7.3 of [Ng-Ma]. [video, scribbles]
• Lecture 21 [Nov 02]: Neural Nets: Backpropagation. Section 7.3 and 7.4 of [Ng-Ma]. [video, scribbles]
• Lab 09 [Nov 04]: Neural Nets
• Lecture 22 [Nov 07]: Convolution Neural Nets. Chapter 14 upto section 14.2.4 of [Murphy]. [video, scribbles]
• Lecture 23 [Nov 09]: Principal Component Analysis. Chapter 12 of [Ng-Ma]. [video, scribbles]
• Lab 10 [Nov 11]: Convolution Neural Nets
• Lecture 24 [Nov 14]: Principal Component Analysis. Chapter 12 of [Ng-Ma] and Appendix C of [Shai²]. [video, scribbles]
• Lecture 25 [Nov 16]: Clustering and k-means. Chapter 10 of [Ng-Ma] and Sections 22.1 and 22.2 of [Shai²]. [video, scribbles]
• Lab 11 [Nov 18]: Principal Component Analysis
• [Nov 19] to [Nov 27]: Fall Break!
• Lecture 26 [Nov 28]: Expectation-Maximization. Section 11.1 of [Ng-Ma]. [video, scribbles]
• Lecture 27 [Nov 30]: Expectation-Maximization. Sections 11.2 through 11.4 of [Ng-Ma]. [video, scribbles]
• Lab 12 [Dec 02]: Clustering and Expectation-Maximization.
• Lecture 28 [Dec 05]: Midterm Review. [video, scribbles]
• Lecture 29 [Dec 07]: Midterm 3
• Final Exam Monday December 12, 8am to 11am.