Mon | Tue | Wed | Thu | Fri |
---|---|---|---|---|
8/24 | 8/25 | 8/26 | 8/27 | 8/28 |
8/31 | 9/1 | 9/2 | 9/3 | 9/4 |
9/7 | 9/8 | 9/9 | 9/10 | 9/11 |
9/14 | 9/15 | 9/16 | 9/17 | 9/18 |
9/21 | 9/22 | 9/23 | 9/24 | 9/25 |
9/28 | 9/29 | 9/30 | 10/1 | 10/2 |
10/5 | 10/6 | 10/7 | 10/8 | 10/9 |
10/12 | 10/13 | 10/14 | 10/15 | 10/16 |
10/19 | 10/20 | 10/21 | 10/22 | 10/23 |
10/26 | 10/27 | 10/28 | 10/29 | 10/30 |
11/2 | 11/3 | 11/4 | 11/5 | 11/6 |
11/9 | 11/10 | 11/11 | 11/12 | 11/13 |
11/16 | 11/17 | 11/18 | 11/19 | 11/20 |
11/23 | 11/24 | 11/25 | 11/26 | 11/27 |
11/30 | 12/1 | 12/2 | 12/3 | 12/4 |
12/7 | 12/8 | 12/9 | 12/10 | 12/11 |
12/14 | 12/15 | 12/16 | 12/17 | 12/18 |
Syllabus: ECE 417
Characteristics of speech and image signals; important analysis and synthesis tools for multimedia signal processing including subspace methods, Bayesian networks, hidden Markov models, and factor graphs; applications to biometrics (person identification), human-computer interaction (face and gesture recognition and synthesis), and audio-visual databases (indexing and retrieval). Emphasis on a set of python machine problems providing hands-on experience. 4 undergraduate hours. 4 graduate hours. Prerequisite: ECE 310 and ECE 313. Assignments | Staff | Resources
Grading Scheme
- 15%: Written homework
- 45%: Machine problems
- 10% each Midterm
- 20% Final Exam
- Up to 1% extra credit for answering the questions of other students on piazza.
Grade cutoffs are approximately as follows, where mu=class average, sigma=standard deviation.
- min(90%, mu+0.25sigma): A-, min(94%, mu+0.5sigma): A, min(99%, mu+1.5sigma): A+
- min(75%, mu-sigma): B-, min(80%, mu-0.75sigma): B, min(85%, mu): B+
- min(60%, mu-2.25sigma): C-, min(65%, mu-2sigma): C, min(70%, mu-1.25sigma): C+
- min(45%, mu-3sigma): D-, min(50%, mu-2.75sigma): D, min(55%, mu-2.5sigma): D+
Assignments
HomeWork
Written homework will be due every two weeks; write your answers by hand, photograph, and submit to Gradescope.
Directory of homework assignments: HW1 | HW2 | HW3 | HW4 | HW5 | HW6
Machine Problems
Machine problems will be in python, once every two weeks, and will be autograded on gradescope.
Directory of machine problems: MP1 | MP2 | MP3 | MP4 | MP5 | MP6
Late Policy
HW and MPs are accepted up to 7 days late on Gradescope, with only a 5% penalty. If you're more than 7 days late, you'll have to submit by e-mail; in general, credit of up to 50% is possible for any submission, at any time before the end of the semester.
Extensions will not be given for software that didn't work; you should have checked that in advance. Extensions are possible in case of illness, on a case-by-case basis.
Academic Integrity
You are encouraged to consult with other students in your attempts to solve any of the MPs. The only thing that’s expressly forbidden is sharing code.
- Do not: share code. Don’t send lines of python by text message or e-mail, don’t post it on your blog, don’t share it on github, don’t post it in piazza.
- Do: consult with other students about how to solve a problem. You can discuss code on a teleconference, as long as nobody's copying the code verbatim. You are encouraged to share pseudo-code and natural language descriptions of algorithms. You are allowed to send chat messages etc. with pseudo-code – just don’t send actual lines of python.
Exams
Exams will be open-book. They will be taken on Compass. They will be timed, and scheduled at the regular lecture time, unless you notify me in advance of a conflict. Sample exams will be available, in advance, for study.
Directory of exams: Midterm 1 | Midterm 2 | Final
Staff
- Professor: Mark Hasegawa-Johnson
- Teaching Assistant: Mahir Morshed
Lectures and Office Hours
Lectures are Tuesdays and Thursdays at 9:30am, at a zoom URL specified on the course Compass page. Lecture videos will be posted to Mediaspace and echo360 after class. If you have trouble accessing the Mediaspace or echo360 video, please send me e-mail.
Office hours will be Fridays and Mondays, 5-6pm, at a zoom URL specified on the course Compass page. You can also ask questions on piazza.
Resources
Readings
Readings will be listed at the top of each HW or MP. These will be published tutorials or research papers.
Software
- Python and NumPy:
Campus Resources
- COVID19
- Statements from the undergraduate advising office about sexual misconduct, academic integrity, religious observances, disability-related accommodations, and FERPA
- Home page of the undergraduate advising office
Lectures, Homework, MPs and Exams
Week 1
- Lecture 1, T8/25 09:30
- Review of DTFT, Gaussians, and Linear Algebra
- Ad for a related course that some of you may find interesting
- Reading: Strang, Section 6.1
- Reading: Gallager, pp. 33-34, 36, 39-43, 45
- Reading: ECE 313 lecture notes
- Lecture 2, R8/27 09:30
- Principal Components and Eigenfaces
- Reading: Gallager, pp. 43-45
- Reading: Eigenfaces for Recognition, Turk and Pentland, 1991
Week 2
- Homework 1, M8/31 23:59
- PDF: Review of DTFT, Gaussians, and Eigenvectors (solutions)
- Lecture 3, T9/1 09:30
- Noise
- Lecture 4, R9/3 09:30
- Filtered Noise
Week 3
- Machine Problem 1, M9/7 23:59
- Assignment web page: PCA
- Lecture 5, T9/8 09:30
- Job posting: Technical manager
- Short Time Fourier Transform
- Lecture 6, R9/10 09:30
- Musical, Perceptual, and Masking Frequency Scales
Week 4
- Homework 2, M9/15 23:59
- Autocorrelation and Power Spectrum: homework, solutions.
- Lecture 7, T9/16 09:30
- Neural Nets
- Lecture 8, T9/18 09:30
- Nonlinearities
Week 5
- Monday 9/21: Machine Problem 2 due at 23:59
- Speech recognition with mel filterbank and gammatone features
- Extra credit: group assignment
- Tuesday 9/22 09:30: Exam 1 Review
- We'll review the practice exams on Compass
- Wednesday 9/23: Extra Office Hours
- Wednesday 9/23 10-11am and 6-7pm, for last-minute exam questions.
- Thursday 9/24: Midterm 1, 09:30-11:00 AM
- The midterm exam will be a timed, open-book, open-notes, open-internet exam, held on Compass. It will appear in your Compass folder at 9:15AM on Thursday 9/24, and will be available until 11:45AM; you may choose any 90-minute period during that time in which to take the exam. You may type your answers in any mixture of plaintext pseudo-math or pseudo-python syntax; as long as we can understand what you mean, you will get the points. Your answer should contain no integrals or infinite-length sums, but otherwise, you do not need to simplify explicit numerical expressions. Examples are available in the two sample exams that are currently available on Compass; you may also find it useful to look at exams from past semesters, though they are in a different format. Reference solutions to all practice exam problems are available on Compass after you submit your answers. Piazza will be open, during the real exam, for private questions to the instructors.
Week 6
- Lecture 9, T9/29 09:30
- Convolutional Neural Networks
- Lecture 10, R10/1 09:30
- Faster-RCNN Object Detectors
Week 7
- Homework 3, M10/5 23:59
- PDF: Back-propagation, loss functions, and nonlinearities. Solutions.
- Lecture 11, T10/6 09:30
- Discriminative vs. Bayesian Classifiers
- Lecture 12, R10/8 09:30
- Hidden Markov Models
Week 8
- Machine Problem 3, M10/12 23:59
- Face detection using a Faster-RCNN object detector
- Lecture 13, T10/13 09:30
- How to train Observation Probability Densities
- Lecture 14, R10/15 09:30
- Log Viterbi and Scaled Forward Algorithms
Week 9
- Homework 4, M10/19 23:59
- Hidden Markov models and Baum-Welch: assignment, solutions
- Lecture 15, T10/20 09:30
- Weighted Finite State Acceptors
- Lecture 16, R10/22 09:30
- Weighted Finite State Transducers
Week 10
- Machine Problem 4, M10/26 23:59
- Assignment page: Speech segmentation using Hidden Markov models
- Midterm Review, T10/27 09:30
- See sample exams on Compass.
- Wednesday 10/28: Extra Office Hours
- Wednesday 10/28 11am-noon and 6-7pm, for last-minute exam questions.
- Midterm 2, R10/29 09:30
- The midterm exam will be a timed, open-book, open-notes, open-internet exam, held on Compass. It will appear in your Compass folder at 9:15AM on Thursday 9/24, and will be available until 12:15PM; you may choose any 90-minute period during that time in which to take the exam. You may type your answers in any mixture of plaintext pseudo-math or pseudo-python syntax; as long as we can understand what you mean, you will get the points. Your answer should contain no integrals or infinite-length sums, but otherwise, you do not need to simplify explicit numerical expressions. Examples are available in the two sample exams that are currently available on Compass; you may also find it useful to look at exams from past semesters, though they are in a different format. Reference solutions to all practice exam problems are available on Compass after you submit your answers. Piazza will be open, during the real exam, for private questions to the instructors.
Week 11
- T 11/3
- No lecture
- R 11/5
- Lecture 17: Practical WFSTs
Week 12
- Homework 5, W11/11 23:59
- Weighted Finite State Transducers: PDF, solutions
- T 11/10
- Lecture 18: Recurrent Neural Networks
- R 11/11
- Lecture 19: Long/Short-Term Memory
Week 13
- Machine Problem 5, W11/18 23:59
- Continuous speech recognition using WFSTs (assignment page)
- T 11/17
- Lecture 20: Moving Pixels Around
- R 11/19
- Lecture 21: Barycentric Coordinates
Week 14
- T 12/1
- Lecture 22: Variational Autoencoders
- Homework 6, W12/2 23:59
- Gated and recurrent neural networks: PDF, solutions.
- R 12/3
- Lecture 23: Generative Adversarial Networks. PDF, Examples
Week 15
- Machine Problem 6, W12/9 23:59
- Video synthesis using LSTMs: Assignment page
Week 16
- Final Exam
- Friday, December 18, 13:30-16:30. If you need a conflict exam because of time zone, or because of a schedule conflict, please e-mail the instructor at least one week prior.
- Sample exam is currently available on Compass.
- The actual final exam will be available from Friday 12/18 at 13:00 until Friday 12/18 at 19:00; you may choose any 180-minute period within that span for your exam.