Inverse problems are central in engineering and science. Most interesting inverse problems are ill-posed and need to be regularized. This course will cover the fundamentals of inverse problems theory including elements from functional analysis, regularization theory, and optimization. After covering the fundamentals of inverse problems, the theory and applications of machine learning to solve inverse problems will be addressed.

A good part of the course will be on the major machine learning and data driven techniques. In particular dictionary learning, transform learning and applications of deep neural networks and generative adversarial networks will be covered.

Tuesday and Thursday, 12:30pm–1:50pm, 2013 ECEB

Instructor: Ivan Dokmanić, dokmanic at illinois dot edu (Office hours: Tuesdays 4:30pm–5:30pm, CSL 313)

Teaching assistant: Sidharth Gupta, gupta67 at illinois dot edu (Office hours: Thursdays 2:30pm–3:30pm, CSL 469B)

January 15 2019: Course begins!

February 12 2019: Homework 1 is available. It is due on 27 February 2019 at 11:59 pm.

February 27 2019: Homework 2 is available. It is due on 12 March 2019 at 11:59 pm.

March 9 2019: Project proposal due on 17 March 2019 at 11:59 pm.

1 page using the template for BASP extended abstracts: http:www.baspfrontiers.org/BASPF

__Abstract__Templates.zip

March 27 2019: Homework 3 is available. It is due on 7 April 2019 at 11:59 pm.

Submission will be via Box. Please contact the teaching assistant if you did not receive a Box invite or have trouble with submitting your work.

Extra credit will be given to typed LaTeX submissions.

Week 1 (1/14–1/18)

Lecture 1: Introduction to inverse problems

Notes: Introduction

Lecture 2: Background in functional analysis

Additional material: Survey paper: Definitions and examples of inverse and ill-posed problems

Additional material: Introduction to Inverse Problems

Week 2 (1/21–1/25)

Lecture 1: Examples of inverse problems

Notes: Examples of inverse problems

Lecture 2: Classical regularization

Python notebook: Deblurring and CT

Week 3 (1/28–2/1)

Lecture 1: Regularization

Notes: Regularizaion and SVD

Lecture 2: Optimization basics

Week 4 (2/4–2/8)

Optimization and Bayesian approaches

Additional material: Scientists Reconstruct an Object by Photographing Its Shadow

Additional material: Computational periscopy with an ordinary digital camera

Additional material: Reconciling “priors” and “priors” without prejudice?

Additional material: Model distortions in Bayesian MAP reconstruction

Additional material: Inverse problems: A Bayesian perspective

Additional material: Convex optimization slides – Boyd and Vandenberghe

Week 5 (2/11–2/15)

Lecture 1: Sampling posteriors

Notes: Sampling posteriors

Lecture 2: Conditional GANs

Notes: Conditional GANs

Related paper: Deep Bayesian Inversion

Week 6 (2/18–2/22)

Lecture 1: Sparsity promoting regularization

Lecture 2: Proximal methods

Notes: Proximal algorithms

Week 7 (2/25–3/1)

Lecture 1: LISTA and Plug-and-Play

Notes: LISTA and Plug-and-Play

Lecture 2: Dual certificates

Notes: Dual certificates

Additional material: Learning Fast Approximations of Sparse Coding

Additional material: Plug-and-Play Priors for Model Based Reconstruction

Week 8 (3/4–3/8)

Lecture 1: Compressed sensing

Notes: Compressed sensing I

Lecture 2: Compressed sensing

Notes: Compressed sensing II

Week 9 (3/11–3/15)

Lecture 1: Wrap-up compressed sensing and introduction to phase retrieval

Lecture 2: Sparse approximation of vectors based on Greed is Good

Additional material: Compressed Sensing using Generative Models

Additional material: Greed is good: Algorithmic results for sparse approximation

Week 10 (3/18–3/22)

Spring break

Week 11 (3/25–3/29)

Lecture 1: Matrix inverse problems

Notes: Matrix inverse problems

Lecture 2: Spectral initialization for matrix inverse problems

Week 12 (4/1–4/5)

Lecture 1: Amortised MAP Inference for Image Super-resolution

Lecture 2: Analyzing Inverse Problems with Invertible Neural Networks

Week 13 (4/8–4/12)

Lecture 1: The Little Engine That Could: Regularization by Denoising (RED)

Lecture 2: Midterm

Week 14 (4/15–4/19)

Lecture 1: On instabilities of deep learning in image reconstruction - Does AI come at a cost?

Lecture 2: Learning Physics with Deep Neural Networks

Week 15 (4/22–4/26)

Lecture 1:

Lecture 2:

40% homeworks

30% one midterm

30% class project

Scherzer, Otmar, Markus Grasmair, Harald Grossauer, Markus Haltmeier, and Frank Lenzen.

*Variational methods in imaging*. Springer Science+ Business Media LLC, 2009.