ECE 544NA: Pattern
Recognition
Course Syllabus, Fall 2013
ECE
544 is a special topics course: lectures and discussions related to advanced
topics and new areas of interest in speech, image, and multidimensional
processing. ECE544NA is the section of this course dedicated to special topics
in pattern recognition. Content varies
every year, but usually includes error metrics (e.g., information-theoretic and
perceptron-based) and optimization (e.g., neural network, Bayesian, stochastic,
and convex programming techniques) for the supervised, semi-supervised, and
unsupervised estimation of probability densities, feature selection, regression
and classification.
In
fall 2013, the course will focus on neural networks, including recent
developments in adaptive and semi-supervised learning of deep networks, as well
as more traditional perceptron-based, density estimation, and
information-theoretic approaches.
Pre-requisites: Vector spaces and probability. For example, it is sufficient to have taken (ECE
313 and ECE 310 or equivalent) or (STAT 542 or equivalent) or (CS 446 or
equivalent).
Text,
fall 2013: Neural
Networks for Pattern Recognition, Christopher Bishop, 1996
The
text will be supplemented occasionally with articles from the professional
literature, e.g., covering the error exponent, covering Boltzmann pre-training,
and covering some of the Bayesian techniques.
Problem sets will not be drawn from the text, so students can use other
texts if desired, but notation in lecture will be drawn primarily from the
Bishop text.
Lecture Topics Contact hours
Bayes’
theorem and the language of pattern recognition |
3 |
Probability
density estimation: non-parametric (kernel-based), parametric (sufficient
stats) |
4.5 |
Linear
classifiers: perceptron, sigmoid, and hinge loss; margin width |
4.5 |
Nonlinear
classifiers: multi-layer and kernel-based |
3 |
Training
criteria: Entropy, error, Bayes error, and the Blahut
error exponent |
3 |
Parameter
optimization methods: convexity, conjugate gradients, expectation maximization |
3 |
Midterm
exam |
1.5 |
Unsupervised
learning: clustering, feature selection, PCA and kernel PCA, Boltzmann
pre-training |
6 |
Generalization:
bias, variance, Vapnik-Chervonenkis dimension |
3 |
Bayesian
techniques: adaptation, transfer learning, hyper-parameter marginalization,
graphical models, structural inference |
7.5 |
Final
project presentations |
6 |
Lecture Total |
43 |
Grading Policy:
Written homework 25%
Matlab
homework 25%
Midterm exam 15%
Final project 35%
Prepared
By:
Mark Hasegawa-Johnson