ECE 544NA: Pattern Recognition

Course Syllabus, Fall 2013

 

ECE 544 is a special topics course: lectures and discussions related to advanced topics and new areas of interest in speech, image, and multidimensional processing. ECE544NA is the section of this course dedicated to special topics in pattern recognition.  Content varies every year, but usually includes error metrics (e.g., information-theoretic and perceptron-based) and optimization (e.g., neural network, Bayesian, stochastic, and convex programming techniques) for the supervised, semi-supervised, and unsupervised estimation of probability densities, feature selection, regression and classification. 

 

In fall 2013, the course will focus on neural networks, including recent developments in adaptive and semi-supervised learning of deep networks, as well as more traditional perceptron-based, density estimation, and information-theoretic approaches.

 

Pre-requisites: Vector spaces and probability.  For example, it is sufficient to have taken (ECE 313 and ECE 310 or equivalent) or (STAT 542 or equivalent) or (CS 446 or equivalent).

 

Text, fall 2013:  Neural Networks for Pattern Recognition, Christopher Bishop, 1996

The text will be supplemented occasionally with articles from the professional literature, e.g., covering the error exponent, covering Boltzmann pre-training, and covering some of the Bayesian techniques.  Problem sets will not be drawn from the text, so students can use other texts if desired, but notation in lecture will be drawn primarily from the Bishop text.

           

Lecture Topics                                                                                                                Contact hours

Bayes’ theorem and the language of pattern recognition   

3

Probability density estimation: non-parametric (kernel-based), parametric (sufficient stats)

4.5

Linear classifiers: perceptron, sigmoid, and hinge loss; margin width

4.5

Nonlinear classifiers: multi-layer and kernel-based

3

Training criteria: Entropy, error, Bayes error, and the Blahut error exponent

3

Parameter optimization methods: convexity, conjugate gradients, expectation maximization

3

Midterm exam

1.5

Unsupervised learning: clustering, feature selection, PCA and kernel PCA, Boltzmann pre-training

6

Generalization: bias, variance, Vapnik-Chervonenkis dimension

3

Bayesian techniques: adaptation, transfer learning, hyper-parameter marginalization, graphical models, structural inference

7.5

Final project presentations

6

Lecture Total

43

 

Grading Policy:

Written homework                   25%

Matlab homework                    25%

Midterm exam                         15%

Final project                             35%

 

Prepared By: Mark Hasegawa-Johnson