Description:
Statistical learning theory is a burgeoning research field at the intersection
of probability, statistics, computer science, and optimization that studies the performance
of computer algorithms for making predictions on the basis of training data.
The following topics will be covered: basics of statistical decision theory;
concentration inequalities; supervised and unsupervised learning; empirical risk minimization;
complexity-regularized estimation; generalization bounds for learning algorithms;
VC dimension and Rademacher complexities; minimax lower bounds; online learning
and optimization. Along with the general theory, we will discuss a number of
applications of statistical learning theory to signal processing, information
theory, and adaptive control.
Problem sets:
Problem sets must be uploaded to compass.
Exams:
Here is a link to Spring 2019 Student Projects
Grading scheme:
Homework, six problem sets (30%), two midterm exams (25% each), project (20%).
Prerequisite:
ECE 534, Random Processes
Credit: 4 graduate hours
Lecture times and location:
TuTh 2:00-3:20 p.m. in Room 2015 ECE Building
Lecture schedule:
schedule of lectures
lecture capture
Assigned Reading:
The reading will mainly be from
notes prepared for this course. These notes will be updated
throughout the semester so it is not recommended that you print them all out.
See the
Fall 2013,
Fall 2014,
Fall 2015,
Spring 2017 , and
Spring 2018
websites for earlier versions of the course. Additional
reading may be assigned from other books or articles, including:
Course Staff and Office Hours:
Question and answer site:
Piazza
About the project:
For the project you are to choose a topic related to the course content and understand and critically evaluate
two or three major papers in that area. Then demonstrate knowledge of the papers by working an example
based on a paper or possibly extending the theory of a paper. You will need to write a project report of five
to ten pages in length, and prepare a fifteen minute presentation.
Additional policy:
Collaboration on the homework is permitted, however each student must write and submit independent solutions. That means working out the final details, the presentation, and wording in the homework solutions on your own. Likewise, while you are requested to not rely on problem sets or notes from previous semesters of this course, if you do find material there or elsewhere, you are still expected to work out the final details, presentation, and wording of the solutions on your own.
Homework is due within the first 5 minutes of the class period on the due date and must be turned in through the compass system. No late homework will be accepted (unless an extension is granted in advance by the instructor).
Problem set 1 .tex solutions
Problem set 2 .tex solutions
Problem set 3 .tex solutions
Problem set 4 .tex solutions
Problem set 5 .tex solutions
Problem set 6 .tex solutions
Exam 1 solutions Here is exam 1 study guide
Exam 2 solutions Here is a summary of second half of the course.
Bruce Hajek, Instructor
b-hajek AT illinois dot edu Office Hours:
Wednesdays, 1-2:30 pm in Room 105 CSL
Bolton Bailey, TA
boltonb2 AT illinois dot edu Office Hours:
Mondays, 3-4 pm in Room 4034 ECEB
Zeyu Zhou, TA
zzhou51 AT illinois dot edu
Office Hours:
Mondays, 5-6 pm in Room 4034 ECEB
Optional recitation sessions.
Staffed by TAs. Fridays, 1pm-2pm in Room 3020 ECEB (north tower)
TAs will work out
sample problems and examples, review background material as needed.
You are required to turn in your homework by uploading it electronically to the compass/blackboard system.
Although you could turn in a VERY NEAT scan of a handwritten version, you
are encouraged to write up your homework solutions in Latex. The .tex source of the problem set is
provided for your convenience. If your solutions are hand written, points may be deducted if the handwriting is difficult to read.
You may bring one sheet of notes to the first exam and two sheets of notes to the second exam. You may use both sides of the sheets,
with font size 10 or larger printing (or similar handwriting size). The examinations are closed book otherwise. Calculators, laptop
computers, tables of integrals, etc. are not permitted.