Computational Photography (CS 445) – Spring
2020
Instructor: Derek Hoiem
Coursera
Syllabus
Take-home
questions
Lecture
Recordings (FA19)
Piazza Discussion Board
Textbook: Computer
Vision: Algorithms and Applications by Rick Szeliski
Projects
Project 1:
Hybrid Images
|
Project
2: Image Quilting
|
Project
3: Gradient Domain Fusion
|
Project 4:
Image-based Lighting
|
Project 5:
Video Stitching
|
Final Project
|
Class Schedule
|
Week |
Date |
Topic |
Link |
Reading/Notes |
|
1 |
Jan 21 (Tues) |
Introduction |
||
|
|
|
Basics of
Working with Images |
|
|
|
1 |
Jan 23 (Thurs) |
Pixels and image filters |
S3.2 (linear filtering) S3.3 (non-linear filtering) |
|
|
2 |
Jan 28 (Tues) |
Thinking in frequency |
S3.4 (fourier transforms) S2.3.3 (compression) |
|
|
2 |
Jan 30 (Thurs) |
Templates and image pyramids |
S3.5.2 (image pyramids) S8.1.1 (pyramid alignment) Other reading: |
|
|
3 |
Feb 4 (Tues) |
Light and color |
S2.2 (light), S2.3.2 (color) or Forsyth and Ponce Ch 3 |
|
|
3 |
Feb 6 (Thurs) |
Histograms and color adjustment |
S3.1 (histograms and color adjustment) |
|
|
|
The Digital
Canvas: Coloring, Blending, Cutting, Synthesizing, and Warping Images |
|
|
|
|
4 |
Feb 11 (Tues) |
Growing: Texture synthesis and hole filling |
Texture
Synthesis – Efros Leung (1999) Image
Quilting – Efros Freeman (2001) |
|
|
4 |
Feb 12
(Wed) |
Project 1
(Hybrid images) due |
|
|
|
4 |
Feb 13 (Thurs) |
Cutting: Intelligent Scissors and Graph Cuts |
||
|
5 |
Feb 18 (Tues) |
Pasting: Compositing and blending |
Poisson
Image Editing – Perez et al. (2003) Burt
and Adelson, A multiresolution spline with
application to image mosaics, ACM ToG (1983) |
|
|
5 |
Feb 20 (Thurs) |
Image warping (translation, rotation, scale, etc.) |
S3.6 (warping) |
|
|
6 |
Feb 25 (Tues) |
Image morphing |
||
|
6 |
Feb 26
(Wed) |
Project 2 (Image quilting) due |
|
|
|
|
|
Modeling
the Physical World |
|
|
|
6 |
Feb 27 (Thurs) |
The Pinhole Camera |
S2.1.5 (3D to 2D projection) |
|
|
7 |
Mar 3 (Tues) |
Single-view Metrology + Camera Model continued |
||
|
7 |
Mar 5 (Thurs) |
Single-view 3D Reconstruction |
Tour into the picture (Horry et
al. 1997) |
|
|
8 |
Mar 7-13 |
Review and
Midterm |
|
|
|
8 |
Mar 18
(Wed) |
Project 3
(Gradient domain fusion) due |
|
|
|
|
Mar 16-20 |
Spring
Break |
|
|
|
9 |
Mar 24 (Tues) |
Image-based Lighting: ray tracing, environment maps, light
probes |
Reading (do read this): |
|
|
9 |
Mar 26 (Thurs) |
Image-based Lighting cont.: HDR light probes, relighting |
Optional
Reading: Debevec & Malik, “Recovering High Dynamic Range
Radiance Maps from Photographs”, SIGGRAPH 1997 Debevec, Rendering Synthetic Objects in Real Scenes, 1998 |
|
|
9 |
Mar 27
(Fri) |
Final
Project Proposal due |
|
|
|
10 |
Mar 31 (Tues) |
The image as a virtual stage |
Rendering
Synthetic Objects (Karsch et al. 2011) |
|
|
|
Correspondence
and Recognition |
|
|
|
|
10 |
Apr 2 (Thurs) |
Matching and alignment with interest points |
Grauman/Leibe Draft Chapter on
Local Features Optional: Lowe - SIFT paper |
|
|
11 |
Apr 7 (Tues) |
Automatic Photo Stitching and RANSAC |
Brown Lowe 2007 ; S9 (stitching); slides |
|
|
|
Apr 8 (Wed) |
Project 4
(Image-based lighting) due |
|
|
|
11 |
Apr 9 (Thurs) |
Object recognition, retrieval, and augmented reality |
||
|
12 |
Apr 14 (Tues) |
Opportunities of scale: texture synthesis, multi-view
reconstruction, im2gps, tiny images, etc. |
Reading: Hays & Efros,
Scene Completion Using Millions of Photographs |
|
|
|
|
More Topics
of Interest |
|
|
|
12 |
Apr 16 (Thurs) |
Generating and detecting fakes |
||
|
13 |
Apr 21 (Tues) |
How the Kinect works |
||
|
|
Apr 22
(Wed) |
Project 5
(Video stitching) due |
|
|
|
13 |
Apr 23 (Thurs) |
Computational approaches to cameras |
Adelson
& Bergen, The Plenoptic Function and the Elements
of Early Vision (1991) Ng
et al., Light Field Photography with a
Hand-held Plenoptic Camera (2005) Levin
et al., Image and Depth with a Coded Aperture (2007) |
|
|
14 |
Apr 28 (Tues) |
Understanding Faces |
|
|
|
14 |
Apr 30 (Thurs) |
Video Magnification |
|
|
|
15 |
May 5 (Tues) |
Last day – cutting edge and wrap up |
|
|
|
15 |
May 8 (Fri) |
Final
Project due |
|
|
|
16 |
May 8-15 |
Final Exam |
|
Credits and Course Notes
I
am grateful for the many preceding efforts to design excellent courses and
course notes for this topic. I am
particularly indebted to Alyosha Efros
for borrowing much of his course and project design, to Rick Szeliski for making his book available, to James Hays for
course suggestions and materials, and to Steve Seitz, Lana Lazebnik,
Silvio Savarese, among others for their slides (as
credited within). Anyone who would
like to use my slides is more than welcome; please do credit the original
sources where appropriate.