Inverse problems are central in engineering and science. Most interesting inverse problems are ill-posed and need to be regularized. This course will cover the fundamentals of inverse problems theory including elements from functional analysis, regularization theory, and optimization. After covering the fundamentals of inverse problems, the theory and applications of machine learning to solve inverse problems will be addressed.
A good part of the course will be on the major machine learning and data driven techniques. In particular dictionary learning, transform learning and applications of deep neural networks and generative adversarial networks will be covered.
Tuesday and Thursday, 12:30pm–1:50pm, 2013 ECEB
Instructor: Ivan Dokmanić, dokmanic at illinois dot edu (Office hours: Tuesdays 4:30pm–5:30pm, CSL 313)
Teaching assistant: Sidharth Gupta, gupta67 at illinois dot edu (Office hours: Thursdays 2:30pm–3:30pm, CSL 469B)
January 15 2019: Course begins!
February 12 2019: Homework 1 is available. It is due on 27 February 2019 at 11:59 pm.
February 27 2019: Homework 2 is available. It is due on 12 March 2019 at 11:59 pm.
March 9 2019: Project proposal due on 17 March 2019 at 11:59 pm.
1 page using the template for BASP extended abstracts: http:www.baspfrontiers.org/BASPFAbstractTemplates.zip
March 27 2019: Homework 3 is available. It is due on 7 April 2019 at 11:59 pm.
Submission will be via Box. Please contact the teaching assistant if you did not receive a Box invite or have trouble with submitting your work.
Extra credit will be given to typed LaTeX submissions.
Week 1 (1/14–1/18)
Lecture 1: Introduction to inverse problems
Notes: Introduction
Lecture 2: Background in functional analysis
Additional material: Survey paper: Definitions and examples of inverse and ill-posed problems
Additional material: Introduction to Inverse Problems
Week 2 (1/21–1/25)
Lecture 1: Examples of inverse problems
Notes: Examples of inverse problems
Lecture 2: Classical regularization
Python notebook: Deblurring and CT
Week 3 (1/28–2/1)
Lecture 1: Regularization
Notes: Regularizaion and SVD
Lecture 2: Optimization basics
Week 4 (2/4–2/8)
Optimization and Bayesian approaches
Additional material: Scientists Reconstruct an Object by Photographing Its Shadow
Additional material: Computational periscopy with an ordinary digital camera
Additional material: Reconciling “priors” and “priors” without prejudice?
Additional material: Model distortions in Bayesian MAP reconstruction
Additional material: Inverse problems: A Bayesian perspective
Additional material: Convex optimization slides – Boyd and Vandenberghe
Week 5 (2/11–2/15)
Lecture 1: Sampling posteriors
Notes: Sampling posteriors
Lecture 2: Conditional GANs
Notes: Conditional GANs
Related paper: Deep Bayesian Inversion
Week 6 (2/18–2/22)
Lecture 1: Sparsity promoting regularization
Lecture 2: Proximal methods
Notes: Proximal algorithms
Week 7 (2/25–3/1)
Lecture 1: LISTA and Plug-and-Play
Notes: LISTA and Plug-and-Play
Lecture 2: Dual certificates
Notes: Dual certificates
Additional material: Learning Fast Approximations of Sparse Coding
Additional material: Plug-and-Play Priors for Model Based Reconstruction
Week 8 (3/4–3/8)
Lecture 1: Compressed sensing
Notes: Compressed sensing I
Lecture 2: Compressed sensing
Notes: Compressed sensing II
Week 9 (3/11–3/15)
Lecture 1: Wrap-up compressed sensing and introduction to phase retrieval
Lecture 2: Sparse approximation of vectors based on Greed is Good
Additional material: Compressed Sensing using Generative Models
Additional material: Greed is good: Algorithmic results for sparse approximation
Week 10 (3/18–3/22)
Spring break
Week 11 (3/25–3/29)
Lecture 1: Matrix inverse problems
Notes: Matrix inverse problems
Lecture 2: Spectral initialization for matrix inverse problems
Week 12 (4/1–4/5)
Lecture 1: Amortised MAP Inference for Image Super-resolution
Lecture 2: Analyzing Inverse Problems with Invertible Neural Networks
Week 13 (4/8–4/12)
Lecture 1: The Little Engine That Could: Regularization by Denoising (RED)
Lecture 2: Midterm
Week 14 (4/15–4/19)
Lecture 1: On instabilities of deep learning in image reconstruction - Does AI come at a cost?
Lecture 2: Learning Physics with Deep Neural Networks
Week 15 (4/22–4/26)
Lecture 1:
Lecture 2:
40% homeworks
30% one midterm
30% class project
Scherzer, Otmar, Markus Grasmair, Harald Grossauer, Markus Haltmeier, and Frank Lenzen. Variational methods in imaging. Springer Science+ Business Media LLC, 2009.