Courses in Machine Learning and Pattern Recognition
The University of Illinois offers a wide variety of courses in machine learning and pattern recognition, distributed across the departments of Computer Science, ECE, and Statistics. These courses are designed to be complementary: few students would wish to take all of these courses, but many students will be interested in taking two or three of them. Course descriptions given here are approximate; official descriptions are available on the UIUC registrar web site. Information similar to this page is maintained more permanently at http://ml.cs.illinois.edu/.
Roughly, these courses can be grouped in three categories.
- Fundamental courses: CS446 | STAT542
- Advanced courses: CS546 | CS 598PS | ECE598MR
- Special topics course: ECE544NA
- CS446: Machine Learning
- STAT542: Statistical Learning
- CS546: Machine Learning and Natural Language
- CS 598PS: Machine Learning for Signal Processing
- ECE 598MR: Statistical Learning Theory
- ECE 544NA: Pattern Recognition
The goal of Machine Learning is to build computer systems that can adapt and learn from their experience. This course will study the theory and application of learning methods that have proved valuable and successful in practical applications. We review the theory of machine learning in order to get a good understanding of the basic issues in this area, and present the main paradigms and techniques needed to obtain successful performance in application areas such as natural language and text understanding, speech recognition, computer vision, data mining, adaptive computer systems and others.
Modern techniques of predictive modeling, classification, and clustering are discussed. Examples of these are linear regression, nonparametric regression, kernel methods, regularization, cluster analysis, classification trees, neural networks, boosting, discrimination, support vector machines, and model selection. Applications are discussed as well as computation and theory. This is probably the only learning/mining course offered outside engineering, and is strongly recommended as an introductory course for non-engineering students interested in the areas of machine learning, statistical learning, and pattern recognition.
Making decisions in natural language processing problems often involves assigning values to sets of interdependent variables where the expressive dependency structure can influence, or even dictate, what assignments are possible. Structured learning problems such as semantic role labeling provide one such example, but the setting is broader and includes a range of problems such as name entity and relation recognition and co-reference resolution. The setting is also appropriate for cases that may require a solution to make use of multiple models (possible pre-designed or pre-learned components) as in summarization, textual entailment and question answering.
There is an increasing need for machines that can understand complex real-world signals, such as speech, images, movies, music, biological and mechanical readings, etc. In this course we will cover the fundamentals of machine learning and signal processing as they pertain to this goal, as well as exciting recent developments. We will learn how to decompose, analyze, classify, detect and consolidate signals, and examine various commonplace operations such as finding faces from camera feeds, organizing personal music collections, designing speech dialog systems and understanding movie content. The course will consist of lectures and student projects and presentations. Students are expected to have a working knowledge of linear algebra, probability theory, and programming skills to carry an implementation of a final project (preferably in MATLAB, but all languages are welcome).
Advanced graduate course on modern probabilistic theory of adaptive and learning systems. The following topics will be covered: basics of statistical decision theory; concentration inequalities; empirical risk minimization; complexity-regularized estimation; generalization bounds for learning algorithms; VC dimension and Rademacher complexities; minimax upper and lower bounds for classification and regression; basics of online learning and optimization. Along with the general theory, the course will discuss applications of statistical learning theory to signal processing, information theory, and adaptive control. Basic prerequisites include probability and random processes, calculus, and linear algebra. Other necessary material and background will be introduced as needed.
ECE544NA is a permanent special-topics course in pattern recognition. Content varies every year, but usually includes error metrics (e.g., information-theoretic and perceptron-based) and optimization (e.g., neural network, Bayesian, stochastic, and convex programming techniques) for the supervised, semi-supervised, and unsupervised estimation of probability densities, feature selection, regression and classification. In fall 2013, ECE 544NA will cover the material usually included in this course, but with a particular emphasis on neural networks. This emphasis on neural networks is prompted by the dramatic success of pre-trained deep belief networks, recently, in many pattern recognition applications.