CS 534 Computational Photography (Fall 2018)
MW 4pm - 5:15pm, EDUC SCI 204

Instructor: Mohit Gupta (mohitg@cs.wisc.edu)  Office Hours: Mon 10am-11am in CS 6395

TA: Zixuan Huang (zhuang356@wisc.edu)    Office Hours: Tues 3pm-5pm in CS 6394

TA: Maulik Shah (maulik@cs.wisc.edu)    Office Hours: Thurs 4pm-6pm in CS 5384

TA: Varun Thumbe (thumbe@wisc.edu)    Office Hours: Fri 3pm-5pm in CS 7352

Discussion Group: Piazza. A discussion for each homework assignment will be created on Piazza. Please post all of your questions on the discussion board so that others may learn from your questions as well. Do not email the professor or TA directly with homework questions.

---

Course Description: Computational photography is an emerging research area whose goal is to design cameras and algorithms in order to capture and create novel visual experiences, images, and videos. This is an inter-disciplinary field spanning several areas, including imaging, image processing, machine learning and computer vision.

This course will provide an introduction to computational photography. Problems in this field include reconstructing the 3D shape of an environment, determining how things are moving, and recognizing people and objects and their activities, all through analysis of images and videos. We will discuss several applications, including creating panoramas, building 3D models, and recognizing faces in images. Along the way, we will learn about basic tools in image processing, imaging and computer vision that enable these applications. Specifically, we will look at topics of image formation, feature detection, motion estimation, image mosaics, 3D shape reconstruction, and object recognition. Applications of these techniques go beyond computational photography, including building 3D maps, creating virtual characters, organizing photo and video databases, human computer interaction, video surveillance, and automatic vehicle navigation. This is a project-based course, in which you will implement several computer vision algorithms and do a final project on a research topic of your choice.

---

Prerequisites
This course will assume a reasonable knowledge of linear algebra and calculus as a prerequisite. The programming assignments will be in MATLAB, so a familiarity with MATLAB is essential.
Please send the instructor an email if you are unsure of whether you can take the course.

Tentative Syllabus

Class Date	Topic	Home-works and Project
W, Sep 5	Introduction and Fun with Optical Illusions	HW-1 out
M, Sep 10	Camera Basics: How Do Images Form?
W, Sep 12	Camera Basics: How Are Images Captured?
M, Sep 17	Binary Images	HW-1 due
W, Sep 19	Image Processing-I – Basic Image Filtering
M, Sep 24	Image Processing-II – Fourier Domain Image Filtering
W, Sep 26	Detecting Edges in Images	Project Proposal due, HW-2 out
M, Oct 1	Hough Transforms: Finding Object Boundaries
W, Oct 3	2D Image Matching Using SIFT
M, Oct 8	Image Alignment and Panorama Creation	HW-2 due, HW-3 out
W, Oct 10	Face Detection
M, Oct 15	Image Segmentation
W, Oct 17	High Dynamic Range Photography and Tone Mapping
M, Oct 22	3D Photography: Photometric Stereo	HW-3 due, HW-4 out
W, Oct 24	No class
M, Oct 29	3D Photography: Camera Focus/Defocus
W, Oct 31	3D Photography: Binocular Stereo	Project Mid-Term Report due
M, Nov 5	3D Photography at City Scale: Leveraging Large Photo Collections
W, Nov 7	Guest Lecture: Prof. Andreas Velten (Cameras that See Around the Corner)	HW-4 due, HW-5 out
M, Nov 12	Guest Lecture: Prof. Yin Li (Deep Learning for Visual Recognition)
W, Nov 14	No class
M, Nov 19	Modern 3D Cameras: Structured Light and Time-of-Flight
W, Nov 21	Measuring Motion and Tracking Objects in Images
M, Nov 26	Multi-Perspective Photography	HW-5 due
W, Nov 28	Computational Cameras
M, Dec 3	Project Presentations
W, Dec 5	Project Presentations
M, Dec 10	Project Presentations
W, Dec 12	Project Presentations	Project Webpage Due

Coursework and Grading

Grading will be based on a 100 point system. There are two main components: (a) Home-works (60% grade), and (b) Final research project (40% grade). Details about these components are given below.

Home-work Assignments

The course will consist of 6 homework assignments. All home-works together will account for 60% of your final grade. Some home-works are lighter than others; their relative weights in your final homework grade are determined by the total number of points listed at the top of each home-work. Guidelines for completing home-work assignments are given here. Please read these carefully.

A discussion for each homework assignment will be created on Piazza. Please post all of your questions on the discussion board so that others may learn from your questions as well. Do not email the professor or TA directly with homework questions.

Include all the files in a zip file named hwX_yourNetID.zip (where X is the homework number) and upload the zip file to Canvas. All home-works are to be submitted by 4 pm on the due date. Only for the home-works (not project), students will be allowed a total of 5 (five) late days per semester; each additional late day will incur a 20% penalty.

Final Research Project

The final project is research-oriented. You are expected to implement one (or more) related research papers, or think of some interesting novel ideas and implement them using the techniques discussed in class. Some possible project ideas are listed here. However, you are welcome to come up with your own ideas - in fact we encourage this. Students can propose their own project topics, subject to the instructor's approval.

Teams of 3 students are strongly preferred. In your submission, please clearly identify the contribution of all the team members. Please note that members in the same group will not necessarily get the same grade.

Project Timeline and What to Submit: There will be three checkpoints: a project proposal, an intermediate milestone report, and a final project webpage. Create a webpage for the project at the beginning. This page will include the proposal, the mid-term report, interesting implementation details, some results (e.g., images, videos, etc.), and so on. Your website should also include downloadable source and executable code. Do not modify the website after the due date of the assignment. Also, send an email to the instructor and TA with the URL of your webpage BEFORE the due dates.

Project Proposal (Due: Sep 26) (5%)

This will be a short report (usually one or two pages will be enough). You will explain what problem you are trying to solve, why you want to solve it, and what are the possible steps to the solution. Please include a time table.

Project Mid-Term Report (Due: Oct 31) (10%)

In this report, you will need to give a brief summary of current progress, including your current results, the difficulties that arise during the implementation, and how your proposal may have changed in light of current progress.

Final Project Presentations (Dec 3, Dec 5, Dec 10, Dec 12) (10%)

This will be a short 'lightning' presentation in class about your project. It will be 5 minutes per team. The slides will be submitted the previous day with automatic timing (you could also submit a video) to ensure that we do not run over. We will collect all the presentations into a single slide-deck that will play automatically. Please add a link to the presentation on the project webpage.

Project Webpage (Due: Dec 12) (15%)

Assemble all the materials you have finished for your project in a webpage or wiki, including the motivation, the approach, your implementation, the results, and discussion of problems encountered. If you have comparison results or some interesting findings, put them on the webpage and discuss these as well.

Readings
The material covered in the course will be available in the presentation slides which will be fairly self-contained. The slides will be made available as printed copies before the start of every class. The slides will not be available electronically. Optional text-book readings include Computer Vision: Algorithms and Applications, by Richard Szeliski. An online version is available, or you may purchase the book at a variety of locations. Also, see Robot Vision by Berthold KP Horn and Optics by E. Hecht.

Acknowledgments
The materials from this class rely significantly on slides prepared by other instructors, especially Shree Nayar, Oliver Cossairt, Peter Belhumeur and Alyosha Efros. Also see other similar courses by Fredo Durand and Bill Freeman (MIT), Peter Belhumeur (Columbia), Irfan Essa (Georgia Tech.), Steve Seitz (U. Washington) and Kyros Kutulakos (U. Toronto). The instructor is extremely thankful to the researchers for making their notes available online.

Academic Integrity
This course follows the University of Wisconsin-Madison Code of Academic Integrity. Unless specifically authorized by the instructor, all coursework is to be done by the student working alone. Violations of the rules will not be tolerated.