Computational Photography (CS 498) – Fall 2012

Instructor: Derek Hoiem (dhoiem at illinois ; Siebel Center 3312)

Office Hours: Mon 10-11am, Thurs 3-4pm + drop-ins

Class Time and Place: Tues and Thurs 11-12:15pm at Siebel Center 1214

Syllabus

Take-home problems

Online Resources

Online Discussion: http://groups.google.com/group/cs498-cp-uiuc

Textbook: Computer Vision: Algorithms and Applications by Rick Szeliski

Matlab: Helpful Tutorials and References ; Writing Fast Code

Projects

Project 1: Hybrid Images Results Favorite Project: JunYoung G. Result: Xinqi C.: Mushroom for Peace	Project 2: Image Quilting Results Favorite Project: Arun M. Result: Nemo (Obama), Jiqin (star night alma mater)	Project 3: Gradient Domain Fusion Results Favorite Project: Jiqin W. Result: Brian W. (Elephant) Nemo (Obama-Orange)
Project 4: Face Morphing Results Favorite Project: Nan Chen Result: Jiqin (How I Grew Up), Nemo (Caricatures)	Project 5: Photo Stitching Results	Final Project

Class Schedule (subject to change)

Date	Topic	Link	Reading/Notes
Aug 28 (Tues)	Introduction	ppt ; pdf
	Basics of Working with Images
Aug 30 (Thurs)	Pixels and image filters	ppt ; pdf	S3.2 (linear filtering) S3.3 (non-linear filtering)
Sept 4 (Tues)	Thinking in frequency	ppt ; pdf	S3.4 (fourier transforms) S2.3.3 (compression) FFT Demo
Sept 5 (Wed)	Matlab Tutorial: 4:30-5:30pm, SC 3403	refsheet examples
Sept 6 (Thurs)	Templates and image pyramids	ppt ; pdf	S3.5.2 (image pyramids) S8.1.1 (pyramid alignment) Other reading: Burt Adelson 1983 – The Laplacian Pyramid
Sept 11 (Tues)	Light and color	ppt ; pdf	S2.2 (light), S2.3.2 (color) or Forsyth and Ponce Ch 3
Sept 11 (Tues)	Linear Algebra Tutorial: 5-6pm, SC 3403	review
Sept 13 (Thurs)	Histograms and color adjustment	ppt ; pdf	S3.1 (histograms and color adjustment) Matlab demo
Sept 17 (Mon)	Project 1 (Hybrid images) due
	The Digital Canvas: Coloring, Blending, Cutting, Synthesizing, and Warping Images
Sept 18 (Tues)	Growing: Texture synthesis and hole filling	ppt ; pdf	Texture Synthesis – Efros Leung (1999) Image Quilting – Efros Freeman (2001) Image Analogies – Hertzmann et al. (2001)
Sept 20 (Thurs)	Cutting: Intelligent Scissors and Graph Cuts	ppt ; pdf	Intelligent Scissors – Mortenson Barret (1995) GrabCut – Rother et al. (2004)
Sept 25 (Tues)	Pasting: Compositing and blending	ppt ; pdf	Project 3 released GradientShop Poisson Image Editing – Perez et al. (2003) Burt and Adelson, A multiresolution spline with application to image mosaics, ACM ToG (1983)
Sept 27 (Thurs)	Image warping (translation, rotation, scale, etc.)	ppt ; pdf	S3.6 (warping)
Oct 1 (Mon)	Project 2 (Image quilting) due
Oct 2 (Tues)	Image morphing	ppt ; pdf	Project 4 released
Oct 4 (Thurs)	The Pinhole Camera	ppt ; pdf	S2.1.5 (3D to 2D projection)
Oct 9 (Tues)	The image as a virtual stage Guest Lecture: David Forsyth		Animation materials Derek out of town Oct 7-14
Oct 11 (Thurs)	PCA and Fun with Faces Guest Lecture: Amin Sadeghi	ppt ; pdf	Derek out of town Oct 7-14
Oct 16 (Tues)	Single-view Metrology	ppt ; pdf	HoiemSavarese-SVG Chapter
Oct 18 (Thurs)	Single-view 3D Reconstruction	ppt ; pdf	Tour into the picture (Horry et al. 1997)
	Working with Photo Collections
Oct 22 (Mon)	Project 3 (Gradient domain fusion) due
Oct 23 (Tues)	Matching and alignment with interest points	ppt ; pdf	Grauman/Leibe Draft Chapter on Local Features Optional: Lowe - SIFT paper
Oct 25 (Thurs)	Automatic Photo Stitching and RANSAC	ppt ; pdf	Project 4 Face Labels Due Brown Lowe 2007 ; S9 (stitching); slides ; Szeliski Tutorial Draft
Oct 30 (Tues)	Object recognition, retrieval, and augmented reality	ppt ; pdf	Grauman/Leibe Draft Visual Vocabularies; S14.3
Nov 1 (Thurs)	Opportunities of scale: texture synthesis, multi-view reconstruction, im2gps, tiny images, etc.	ppt ; pdf	Reading: Hays & Efros, Scene Completion Using Millions of Photographs
Nov 5 (Mon)	Project 4 (Face morphing) due
Nov 6 (Tues)	Midterm Review	ppt ; pdf	Project 5 released
Nov 8 (Thurs)	Midterm Exam, normal time/place
	More Topics of Interest
Nov 13 (Tues)	Detecting fakes	ppt ; pdf
Nov 15 (Thurs)	NO CLASS (work on project 5 and final project)		Derek in Portland: Nov 14-16
Nov 20, 22	NO CLASS - Thanksgiving Break!
Nov 26 (Mon)	Project 5 (Image stitching) due
Nov 27 (Tues)	Image-based Lighting: ray tracing, environment maps, light probes	ppt ; pdf	Reading (do read this): Debevec, Image-based Lighting Tutorial (2005)
Nov 29 (Thurs)	Image-based Lighting cont.: HDR light probes, relighting	ppt ; pdf	Optional Reading: Debevec & Malik, “Recovering High Dynamic Range Radiance Maps from Photographs”, SIGGRAPH 1997 Debevec, Rendering Synthetic Objects in Real Scenes, 1998
Dec 3 (Mon)	Final Project Proposal due		Derek in Pittsburgh: Dec 3
Dec 4 (Tues)	Computational approaches to cameras	ppt ; pdf
Dec 6 (Thurs)	How the Kinect works	ppt ; pdf
Dec 11 (Tues)	Last day – wrap up	ppt ; pdf	Derek in Pittsburgh: Dec 11-13
Dec 18 (Tues), 8am-11am	Final Project Presentations (SC1214)

Some other ideas for special topics:

Students, let me know if there’s something you’d especially like to cover.

Some ideas: 1) Background subtraction and alpha matting; 2) Special or Programmable cameras; 3) Environment maps and image-based lighting; 4) What makes a good (or real) photo?; 5) Video textures; 6) Recoloring; 7) Tricks with focus or aperture (e.g. creating HDR images from multiple exposures); 8) Physics-based models (modeling fog, water, etc.); 9) Deconvolution and deblurring; 10) superresolution; 11) Non-photo realistic rendering; 12) Kinect sensor and applications

Similar Courses in Other Universities

Computational Photography (Efros, CMU)

Computational Photography (Hays, Brown)

Computational Photography SIGGRAPH Course (Raskar & Tumblin)

Computational Camera and Photography (Raskar, MIT Media Lab)

Digital and Computational Photography (Durand & Freeman, MIT)

Computational Photography (Essa, Georgia Tech)

Computational Photography (Levoy, Adams, Pulli, Stanford)

Computational Photography (Lazebnik, UNC)

Computational Photography (Fergus, NYU)

Internet Vision (T.Berg, SUNY)

Computer Vision (Seitz & Szeliski, UWashington)

Introduction to Visual Computing and Visual Modeling (Kutulakos, UToronto)

Symposium on Computational Photography and Video (May 2005, MIT)

Credits and Course Notes

I am grateful for the many preceding efforts to design excellent courses and course notes for this topic. I am particularly indebted to Alyosha Efros for borrowing much of his course and project design, to Rick Szeliski for making his book available, to James Hays for course suggestions and materials, and to Steve Seitz, Lana Lazebnik, Silvio Savarese, among others for their slides (as credited within). Anyone who would like to use my slides is more than welcome; please do credit the original sources where appropriate.