Computational Photography (CS 445) – Fall 2019

Instructor: Derek Hoiem (dhoiem)

TAs: Wilfredo Torres Calderon (trrscld2)

Yuan Shen (yshen47)

Jae Yong Lee (lee896)

Office Hours:

1 hour, held near Siebel 3407

Mondays 11:30AM (Derek), now in his office 3312

Tuesdays 7PM (Jae)

Thursdays 5PM (Yuan)

Fridays 11AM (Wilfredo)

Class Time and Place: Wed and Fri 2:00-3:30 at DCL 1320

Syllabus

Take-home questions

Lecture Recordings

Homework Submission (Compass)

Piazza Discussion Board

Textbook: Computer Vision: Algorithms and Applications by Rick Szeliski

Projects

Project 1: Hybrid Images Results	Project 2: Image Quilting Results	Project 3: Gradient Domain Fusion Results
Project 4: Image-based Lighting Results	Project 5: Video Stitching Results	Final Project Results

Class Schedule (subject to change)

Date	Topic	Link	Reading/Notes
Aug 28 (Wed)	Introduction	ppt ; pdf
	Basics of Working with Images
Aug 30 (Fri)	Pixels and image filters	ppt ; pdf	S3.2 (linear filtering) S3.3 (non-linear filtering)
Sept 4 (Wed)	Thinking in frequency Derek away Sept 1-5. Lecture by Jae.	ppt ; pdf ’15 video	S3.4 (fourier transforms) S2.3.3 (compression) FFT Demo (original) ; Python demo by Jae
Sept 5 (Thurs)	Linear Algebra Tutorial at 5-6:50PM in DCL 1320 by Yuan. Use of Jupyter Notebook covered at start.	review ; demo	Another linear algebra review
Sept 6 (Fri)	Templates and image pyramids	ppt ; pdf	S3.5.2 (image pyramids) S8.1.1 (pyramid alignment) Other reading: Burt Adelson 1983 – The Laplacian Pyramid
Sept 11 (Wed)	Light and color	ppt ; pdf	S2.2 (light), S2.3.2 (color) or Forsyth and Ponce Ch 3
Sept 13 (Fri)	Histograms and color adjustment	ppt ; pdf	S3.1 (histograms and color adjustment) Color balance demo
Sept 17 (Tues)	Project 1 (Hybrid images) due
	The Digital Canvas: Coloring, Blending, Cutting, Synthesizing, and Warping Images
Sept 18 (Wed)	Growing: Texture synthesis and hole filling	ppt ; pdf	Texture Synthesis – Efros Leung (1999) Image Quilting – Efros Freeman (2001) Image Analogies – Hertzmann et al. (2001) Patch Match – Barnes et al. (2009)
Sept 20 (Fri)	CS 497 only: Team and Topic Proposal
Sept 20 (Fri)	Cutting: Intelligent Scissors and Graph Cuts	ppt ; pdf	Intelligent Scissors – Mortenson Barret (1995) GrabCut – Rother et al. (2004)
Sept 25 (Wed)	Pasting: Compositing and blending	ppt ; pdf	Poisson Image Editing – Perez et al. (2003) Burt and Adelson, A multiresolution spline with application to image mosaics, ACM ToG (1983)
Sept 27 (Fri)	Image warping (translation, rotation, scale, etc.)	ppt ; pdf	S3.6 (warping) Alignment demo
Oct 1 (Tues)	Project 2 (Image quilting) due
Oct 2 (Wed)	Image morphing	ppt ; pdf
	Modeling the Physical World
Oct 4 (Fri)	The Pinhole Camera	ppt ; pdf	S2.1.5 (3D to 2D projection)
Oct 9 (Wed)	Single-view Metrology + Camera Model continued	ppt ; pdf	HoiemSavarese-SVG Chapter
Oct 11 (Fri)	Single-view 3D Reconstruction	ppt ; pdf	Tour into the picture (Horry et al. 1997)
Oct 15 (Tues)	Project 3 (Gradient domain fusion) due
Oct 16 (Wed)	Image-based Lighting: ray tracing, environment maps, light probes	ppt ; pdf	Reading (do read this): Debevec, Image-based Lighting Tutorial (2005)
Oct 18 (Fri)	Image-based Lighting cont.: HDR light probes, relighting	ppt ; pdf	Optional Reading: Debevec & Malik, “Recovering High Dynamic Range Radiance Maps from Photographs”, SIGGRAPH 1997 Debevec, Rendering Synthetic Objects in Real Scenes, 1998
Oct 18 (Fri)	CS 497 only: Detailed Project Plan
Oct 23 (Wed)	The image as a virtual stage	ppt ; pdf	Rendering Synthetic Objects (Karsch et al. 2011)
	Correspondence and Recognition
Oct 25 (Fri)	Matching and alignment with interest points	ppt ; pdf	Grauman/Leibe Draft Chapter on Local Features Optional: Lowe - SIFT paper
Oct 29 (Tues)	Project 4 (Image-based lighting) due
Oct 30 (Wed)	Automatic Photo Stitching and RANSAC	ppt ; pdf	Brown Lowe 2007 ; S9 (stitching); slides
Nov 1 (Fri)	Object recognition, retrieval, and augmented reality	ppt ; pdf	Grauman/Leibe Draft Visual Vocabularies; S14.3
Nov 6 (Wed)	Opportunities of scale: texture synthesis, multi-view reconstruction, im2gps, tiny images, etc.	ppt ; pdf	Reading: Hays & Efros, Scene Completion Using Millions of Photographs
	More Topics of Interest
Nov 8 (Fri)	Generating and detecting fakes	ppt ; pdf	Pix2pix Cycle GAN Deep fakes in news
Nov 12 (Tues)	Project 5 (Video stitching) due
Nov 13 (Wed)	How the Kinect works	ppt ; pdf
Nov 15 (Fri)	Computational approaches to cameras	ppt ; pdf	Adelson & Bergen, The Plenoptic Function and the Elements of Early Vision (1991) Ng et al., Light Field Photography with a Hand-held Plenoptic Camera (2005) Levin et al., Image and Depth with a Coded Aperture (2007)
Nov 18 (Mon)	Final Project Proposal due (for CS 497 students, this is a midpoint report)
Nov 20 (Wed)	Exam Review	ppt ; pdf	Example questions
Nov 22 (Fri)	Exam, normal class time/place
Nov 27, 29	NO CLASS - Thanksgiving Break!
Dec 4 (Wed)	Understanding Faces	ppt ; pdf
Dec 6 (Fri)	Video Magnification	ppt ; pdf
Dec 11 (Wed)	Last day – cutting edge and wrap up	ppt ; pdf
Dec 16 8-11am (Mon)	Final Poster Presentations (final exam period). In Siebel Center 1^st floor atrium

Credits and Course Notes

I am grateful for the many preceding efforts to design excellent courses and course notes for this topic. I am particularly indebted to Alyosha Efros for borrowing much of his course and project design, to Rick Szeliski for making his book available, to James Hays for course suggestions and materials, and to Steve Seitz, Lana Lazebnik, Silvio Savarese, among others for their slides (as credited within). Anyone who would like to use my slides is more than welcome; please do credit the original sources where appropriate.