Mon Tue Wed Thu Fri
8/24 8/25 8/26 8/27 8/28
8/31 9/1 9/2 9/3 9/4
9/7 9/8 9/9 9/10 9/11
9/14 9/15 9/16 9/17 9/18
9/21 9/22 9/23 9/24 9/25
9/28 9/29 9/30 10/1 10/2
10/5 10/6 10/7 10/8 10/9
10/12 10/13 10/14 10/15 10/16
10/19 10/20 10/21 10/22 10/23
10/26 10/27 10/28 10/29 10/30
11/2 11/3 11/4 11/5 11/6
11/9 11/10 11/11 11/12 11/13
11/16 11/17 11/18 11/19 11/20
11/23 11/24 11/25 11/26 11/27
11/30 12/1 12/2 12/3 12/4
12/7 12/8 12/9 12/10 12/11
12/14 12/15 12/16 12/17 12/18

Syllabus: ECE 417

Characteristics of speech and image signals; important analysis and synthesis tools for multimedia signal processing including subspace methods, Bayesian networks, hidden Markov models, and factor graphs; applications to biometrics (person identification), human-computer interaction (face and gesture recognition and synthesis), and audio-visual databases (indexing and retrieval). Emphasis on a set of python machine problems providing hands-on experience. 4 undergraduate hours. 4 graduate hours. Prerequisite: ECE 310 and ECE 313. Assignments | Staff | Resources

Grading Scheme

Grade cutoffs are approximately as follows, where mu=class average, sigma=standard deviation.



Written homework will be due every two weeks; write your answers by hand, photograph, and submit to Gradescope.

Directory of homework assignments: HW1 | HW2 | HW3 | HW4 | HW5 | HW6

Machine Problems

Machine problems will be in python, once every two weeks, and will be autograded on gradescope.

Directory of machine problems: MP1 | MP2 | MP3 | MP4 | MP5 | MP6

Late Policy

HW and MPs are accepted up to 7 days late on Gradescope, with only a 5% penalty. If you're more than 7 days late, you'll have to submit by e-mail; in general, credit of up to 50% is possible for any submission, at any time before the end of the semester.

Extensions will not be given for software that didn't work; you should have checked that in advance. Extensions are possible in case of illness, on a case-by-case basis.

Academic Integrity

You are encouraged to consult with other students in your attempts to solve any of the MPs. The only thing that’s expressly forbidden is sharing code.


Exams will be open-book. They will be taken on Compass. They will be timed, and scheduled at the regular lecture time, unless you notify me in advance of a conflict. Sample exams will be available, in advance, for study.

Directory of exams: Midterm 1 | Midterm 2 | Final


Lectures and Office Hours

Lectures are Tuesdays and Thursdays at 9:30am, at a zoom URL specified on the course Compass page. Lecture videos will be posted to Mediaspace and echo360 after class. If you have trouble accessing the Mediaspace or echo360 video, please send me e-mail.

Office hours will be Fridays and Mondays, 5-6pm, at a zoom URL specified on the course Compass page. You can also ask questions on piazza.



Readings will be listed at the top of each HW or MP. These will be published tutorials or research papers.


Campus Resources

Lectures, Homework, MPs and Exams

Week 1

Lecture 1, T8/25 09:30
Review of DTFT, Gaussians, and Linear Algebra
Ad for a related course that some of you may find interesting
Reading: Strang, Section 6.1
Reading: Gallager, pp. 33-34, 36, 39-43, 45
Reading: ECE 313 lecture notes
Lecture 2, R8/27 09:30
Principal Components and Eigenfaces
Reading: Gallager, pp. 43-45
Reading: Eigenfaces for Recognition, Turk and Pentland, 1991

Week 2

Homework 1, M8/31 23:59
PDF: Review of DTFT, Gaussians, and Eigenvectors (solutions)
Lecture 3, T9/1 09:30
Lecture 4, R9/3 09:30
Filtered Noise

Week 3

Machine Problem 1, M9/7 23:59
Assignment web page: PCA
Lecture 5, T9/8 09:30
Job posting: Technical manager
Short Time Fourier Transform
Lecture 6, R9/10 09:30
Musical, Perceptual, and Masking Frequency Scales

Week 4

Homework 2, M9/15 23:59
Autocorrelation and Power Spectrum: homework, solutions.
Lecture 7, T9/16 09:30
Neural Nets
Lecture 8, T9/18 09:30

Week 5

Monday 9/21: Machine Problem 2 due at 23:59
Speech recognition with mel filterbank and gammatone features
Extra credit: group assignment
Tuesday 9/22 09:30: Exam 1 Review
We'll review the practice exams on Compass
Wednesday 9/23: Extra Office Hours
Wednesday 9/23 10-11am and 6-7pm, for last-minute exam questions.
Thursday 9/24: Midterm 1, 09:30-11:00 AM
The midterm exam will be a timed, open-book, open-notes, open-internet exam, held on Compass. It will appear in your Compass folder at 9:15AM on Thursday 9/24, and will be available until 11:45AM; you may choose any 90-minute period during that time in which to take the exam. You may type your answers in any mixture of plaintext pseudo-math or pseudo-python syntax; as long as we can understand what you mean, you will get the points. Your answer should contain no integrals or infinite-length sums, but otherwise, you do not need to simplify explicit numerical expressions. Examples are available in the two sample exams that are currently available on Compass; you may also find it useful to look at exams from past semesters, though they are in a different format. Reference solutions to all practice exam problems are available on Compass after you submit your answers, and will also be posted on the course web page on Tuesday 9/22 after lecture. Piazza is open now for questions about the practice exam, and will be open, during the real exam, for private questions to the instructors.

Week 6

Lecture 9, T9/29 09:30
Convolutional Neural Networks
Lecture 10, R10/1 09:30
Faster-RCNN Object Detectors

Week 7

Homework 3, M10/5 23:59
PDF: Back-propagation, loss functions, and nonlinearities

Week 8

Machine Problem 3, M10/12 23:59
Face detection using a Faster-RCNN object detector

Week 9

Homework 4, M10/19 23:59
Hidden Markov models and Baum-Welch

Week 10

Machine Problem 4, M10/26 23:59
Isolated word recognition using Hidden Markov models
Midterm 2, R10/29 09:30
Details will show up here.

Week 12

Homework 5, W11/11 23:59
Weighted Finite State Transducers

Week 13

Machine Problem 5, W11/18 23:59
Continuous speech recognition using HMMs and WFSTs

Week 14

Homework 6, W12/2 23:59
Gated and recurrent neural networks

Week 15

Machine Problem 6, W12/9 23:59
Video synthesis using LSTMs

Week 16?

Final Exam
Date TBA