Computational Photography (CS 445) – Spring 2020  

  

Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: <i>The Key to the Fields</i> (Magritte 1936)

  Instructor:  Derek Hoiem

 

 

 

  Coursera

  Syllabus

  Take-home questions

  Lecture Recordings (FA19)

  Piazza Discussion Board

 

 

 

 

  Textbook: Computer Vision: Algorithms and Applications by Rick Szeliski

                                                                                                           

  

   Projects

Project 1: Hybrid Images

Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: leopard elephant

Project 2: Image Quilting

    Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: C:\Users\Hoiem\Documents\Classes\ComputationalPhotography - Fall 2012\projects\quilting\face_toast_cropped.png

   

Project 3: Gradient Domain Fusion

Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: cloning_teaser

 

Project 4: Image-based Lighting

https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcQJ6tcY1JugjgyMMpZMj9BgU6OaGyIv8jL4T8J-Qsgnn0UOrHT2tw

 

 

Project 5: Video Stitching

 

 

Final Project

Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: http://t2.gstatic.com/images?q=tbn:ANd9GcSEXEDpOJHSX2MOS0mvolJmNhoGKiQZ6t4ky5dyEio1mph1-L1x

 

 

 

  

  Class Schedule  

Week

Date

Topic

Link

Reading/Notes

1

Jan 21 (Tues)

Introduction

ppt ; pdf ; vid

 

 

Basics of Working with Images

 

 

1

Jan 23 (Thurs)

Pixels and image filters

ppt ; pdf ; vid

S3.2 (linear filtering)

S3.3 (non-linear filtering)

2

Jan 28 (Tues)

Thinking in frequency

ppt ; pdf ; vid

S3.4 (fourier transforms)

S2.3.3 (compression)

FFT Demo (original) ; Python demo by Jae

2

Jan 30 (Thurs)

Templates and image pyramids

ppt ; pdf ; vid

S3.5.2 (image pyramids)

S8.1.1 (pyramid alignment)

Other reading:

   Burt Adelson 1983 – The Laplacian Pyramid

3

Feb 4 (Tues)

Light and color

ppt ; pdf ; vid

S2.2 (light), S2.3.2 (color)

or Forsyth and Ponce Ch 3

3

Feb 6 (Thurs)

Histograms and color adjustment

ppt ; pdf ; vid

S3.1 (histograms and color adjustment)

Color balance demo

 

The Digital Canvas: Coloring, Blending, Cutting, Synthesizing, and Warping Images

 

 

4

Feb 11 (Tues)

Growing: Texture synthesis and hole filling

ppt ; pdf

Texture Synthesis – Efros Leung (1999)

Image Quilting – Efros Freeman (2001)

Image Analogies – Hertzmann et al. (2001)

Patch Match – Barnes et al. (2009)

4

Feb 12 (Wed)

Project 1 (Hybrid images) due

 

 

4

Feb 13 (Thurs)

Cutting: Intelligent Scissors and Graph Cuts

ppt ; pdf

Intelligent Scissors – Mortenson Barret (1995)

GrabCut – Rother et al. (2004)

5

Feb 18 (Tues)

Pasting: Compositing and blending

ppt ; pdf

Poisson Image Editing – Perez et al. (2003)

Burt and Adelson, A multiresolution spline with application to image mosaics, ACM ToG (1983) 

5

Feb 20 (Thurs)

Image warping (translation, rotation, scale, etc.)

ppt ; pdf

S3.6 (warping)

Alignment demo

6

Feb 25 (Tues)

Image morphing

ppt ; pdf

6

Feb 26 (Wed)

Project 2 (Image quilting) due

 

 

 

 

Modeling the Physical World

 

 

6

Feb 27 (Thurs)

The Pinhole Camera

ppt ; pdf

S2.1.5 (3D to 2D projection)

7

Mar 3 (Tues)

Single-view Metrology + Camera Model continued

ppt ; pdf

HoiemSavarese-SVG Chapter

7

Mar 5 (Thurs)

Single-view 3D Reconstruction

ppt ; pdf

Tour into the picture (Horry et al. 1997)

8

Mar 11 (Wed)

Project 3 (Gradient domain fusion) due

 

 

8

Mar 7-13

Review and Midterm

 

 

 

Mar 16-20

Spring Break

 

 

9

Mar 24 (Tues)

Image-based Lighting: ray tracing, environment maps, light probes

ppt ; pdf

Reading (do read this):

Debevec, Image-based Lighting Tutorial (2005)

9

Mar 26 (Thurs)

Image-based Lighting cont.: HDR light probes, relighting

ppt ; pdf

Optional Reading:

Debevec & Malik, “Recovering High Dynamic Range Radiance Maps from Photographs”, SIGGRAPH 1997

Debevec, Rendering Synthetic Objects in Real Scenes, 1998

9

Mar 27 (Fri)

Final Project Proposal due

 

 

10

Mar 31 (Tues)

The image as a virtual stage

ppt ; pdf

Rendering Synthetic Objects (Karsch et al. 2011)

 

Correspondence and Recognition

 

 

10

Apr 2  (Thurs)

Matching and alignment with interest points

ppt ; pdf

Grauman/Leibe Draft Chapter on Local Features Optional: Lowe - SIFT paper

11

Apr 7 (Tues)

Automatic Photo Stitching and RANSAC

ppt ; pdf

Brown Lowe 2007 ; S9 (stitching);  slides

 

Apr 8 (Wed)

Project 4 (Image-based lighting) due

 

 

11

Apr 9 (Thurs)

Object recognition, retrieval, and augmented reality

ppt ; pdf

Grauman/Leibe Draft Visual Vocabularies; S14.3

12

Apr 14 (Tues)

Opportunities of scale: texture synthesis, multi-view reconstruction, im2gps, tiny images, etc.

ppt ; pdf

Reading: Hays & Efros, Scene Completion Using Millions of Photographs

 

 

More Topics of Interest

 

 

12

Apr 16 (Thurs)

Generating and detecting fakes  

ppt ; pdf

Pix2pix

Cycle GAN

Deep fakes in news

13

Apr 21 (Tues)

How the Kinect works

ppt ; pdf

 

Apr 22 (Wed)

Project 5 (Video stitching) due

 

 

13

Apr 23 (Thurs)

Computational approaches to cameras

ppt ; pdf

Adelson & Bergen, The Plenoptic Function and the Elements of Early Vision (1991)

Ng et al., Light Field Photography with a Hand-held Plenoptic Camera (2005)

Levin et al., Image and Depth with a Coded Aperture (2007)

14

Apr 28 (Tues)

Understanding Faces

ppt ; pdf

 

14

Apr 30 (Thurs)

Video Magnification

ppt ; pdf

 

15

May 5 (Tues)

Last day – cutting edge and wrap up

ppt ; pdf

 

15

May 8 (Fri)

Final Project due

 

 

16

May 8-15

Final Exam

 

 





 


 

 

 

Credits and Course Notes

I am grateful for the many preceding efforts to design excellent courses and course notes for this topic.  I am particularly indebted to Alyosha Efros for borrowing much of his course and project design, to Rick Szeliski for making his book available, to James Hays for course suggestions and materials, and to Steve Seitz, Lana Lazebnik, Silvio Savarese, among others for their slides (as credited within).  Anyone who would like to use my slides is more than welcome; please do credit the original sources where appropriate.


Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: GoStats.com