Computational Photography (CS 445) – Spring
2021
Instructor: Derek Hoiem
Coursera
Syllabus
Take-home
questions
Original Lecture
Recordings (FA19)
Piazza Discussion
Board
Textbook: Computer Vision: Algorithms and Applications
by Rick Szeliski
Projects
Project 1:
Hybrid Images
|
Project
2: Image Quilting
|
Project
3: Gradient Domain Fusion
|
Project 4:
Image-based Lighting
|
Project 5:
Video Stitching
|
Final Project
|
Class Schedule
|
Week |
Date |
Topic |
Link |
Reading/Notes |
|
1 |
Jan 25 – Jan 31 |
Introduction |
||
|
|
|
Basics of
Working with Images |
|
|
|
1 |
Pixels and image filters |
S3.2 (linear filtering) S3.3 (non-linear filtering) |
||
|
2 |
Feb 1 – Feb 7 |
Thinking in frequency |
S3.4 (fourier transforms) S2.3.3 (compression) |
|
|
2 |
Templates and image pyramids |
S3.5.2 (image pyramids) S8.1.1 (pyramid alignment) Other reading: |
||
|
3 |
Feb 8 – Feb 14 |
Light and color |
S2.2 (light), S2.3.2 (color) or Forsyth and Ponce Ch 3 |
|
|
3 |
Histograms and color adjustment |
S3.1 (histograms and color adjustment) |
||
|
|
Feb 15
(Mon) |
Project 1
(Hybrid images) due |
|
|
|
|
The Digital
Canvas: Coloring, Blending, Cutting, Synthesizing, and Warping Images |
|
|
|
|
4 |
Feb 15 – Feb 21 (Feb 17 break) |
Growing: Texture synthesis and hole filling |
Texture
Synthesis – Efros Leung (1999) Image
Quilting – Efros Freeman (2001) |
|
|
4 |
Cutting: Intelligent Scissors and Graph Cuts |
|||
|
5 |
Feb 22 – Feb 28 |
Pasting: Compositing and blending |
Poisson
Image Editing – Perez et al. (2003) Burt
and Adelson, A multiresolution spline with
application to image mosaics, ACM ToG (1983) |
|
|
5 |
Image warping (translation, rotation, scale, etc.) |
S3.6 (warping) |
||
|
|
Mar 1 (Mon) |
Project 2 (Image quilting) due |
|
|
|
6 |
Mar 1 – Mar 7 |
Image morphing |
||
|
|
|
Modeling the
Physical World |
|
|
|
6 |
The Pinhole Camera |
S2.1.5 (3D to 2D projection) |
||
|
7 |
Mar 8 – Mar 14 |
Single-view Metrology + Camera Model continued |
||
|
7 |
Single-view 3D Reconstruction |
Tour into the picture (Horry et
al. 1997) |
||
|
8 |
Mar 15
– Mar 21 |
Midterm
Exam (on PrarieLearn) |
|
|
|
9 |
Mar 22 – Mar 28 (Mar 24 break) |
Image-based Lighting: ray tracing, environment maps, light
probes |
Reading (do read this): |
|
|
9 |
Image-based Lighting cont.: HDR light probes, relighting |
Optional
Reading: Debevec & Malik, “Recovering High Dynamic Range
Radiance Maps from Photographs”, SIGGRAPH 1997 Debevec, Rendering Synthetic Objects in Real Scenes, 1998 |
||
|
|
Mar 29 |
Project 3
(Gradient domain fusion) due |
|
|
|
10 |
Mar 29 – Apr 4 |
The image as a virtual stage |
Rendering
Synthetic Objects (Karsch et al. 2011) |
|
|
|
Correspondence
and Recognition |
|
|
|
|
10 |
Matching and alignment with interest points |
Grauman/Leibe Draft Chapter on
Local Features Optional: Lowe - SIFT paper |
||
|
|
Apr 5 |
Final
Project Proposal due |
|
|
|
11 |
Apr 5 – Apr 11 |
Automatic Photo Stitching and RANSAC |
Brown Lowe 2007 ; S9 (stitching); slides |
|
|
11 |
Object recognition, retrieval, and augmented reality |
|||
|
|
Apr 12
(Mon) |
Project 4
(Image-based lighting) due |
|
|
|
12 |
Apr 12 – Apr 18 (Apr 13 break) |
Opportunities of scale: texture synthesis, multi-view
reconstruction, im2gps, tiny images, etc. |
Reading: Hays & Efros,
Scene Completion Using Millions of Photographs |
|
|
|
|
More Topics
of Interest |
|
|
|
12 |
Generating and detecting fakes |
|||
|
13 |
Apr 19 – Apr 25 |
How the Kinect works |
||
|
13 |
Computational approaches to cameras |
Adelson
& Bergen, The Plenoptic Function and the Elements
of Early Vision (1991) Ng
et al., Light Field Photography with a
Hand-held Plenoptic Camera (2005) Levin
et al., Image and Depth with a Coded Aperture (2007) |
||
|
|
Apr 26
(Mon) |
Project 5
(Video stitching) due |
|
|
|
14 |
Apr 26 – May 2 |
Understanding Faces |
|
|
|
14 |
Video Magnification |
|
||
|
|
|
|
|
|
|
15 |
May 3 – 5 |
Last day – cutting edge and wrap up |
|
|
|
May 9 (Sun) |
Final
Project due (cannot be late) |
|
|
|
|
16 |
May 7 – May 14 |
Final Exam
(on PrairieLearn) |
|
Credits and Course Notes
I
am grateful for the many preceding efforts to design excellent courses and
course notes for this topic. I am
particularly indebted to Alyosha Efros
for borrowing much of his course and project design, to Rick Szeliski for making his book available, to James Hays for
course suggestions and materials, and to Steve Seitz, Lana Lazebnik, Silvio
Savarese, among others for their slides (as credited within). Anyone who would like to use my slides
is more than welcome; please do credit the original sources where appropriate.