CS 534: Computational Photography, University of Wisconsin

HW #5 Projects

Kevin Atherton, Ted Imhoff-Smith and Zachary Ovanin
Object Detection and Censoring: The Removal of Unwanted Artifacts in Social Photography
Methods for object detection and segmentation as well as image completion have been investigated in prior research. Although effective methods for object detection and image completion have been developed, research has not been conducted to detect and remove unwanted or potentially obscene objects from images in the context of social networking without necessitating human input. Our aim in our project is to create an automatic method for pruning images of unwanted or unnoticed objects, or in our specific implementation, alcohol containers in Facebook images. Combining object recognition and image replacement techniques will allow for automatic censorship of unwanted or obscene objects within an image. This project implements an effective method for unwanted artifact detection and elimination using a combination of SIFT feature point detection and image replacement methods, demonstrating strong potential for application in social networking contexts such as with Facebook photo upload applications.
Project webpage
Ali Bramson and Tom Finzell
Making New Worlds
This paper describes an algorithm for generating new planets. These newly formed fictional planets could be used as the backdrops of science fiction movies and games. Starting with a preexisting image of a planet, it builds off of it by incorporating elements of other planets and adjusting the intensity values of the colormap. We start with template RGB images of actual planets in our solar system. We select surface features (i.e. mountains, lakes, craters) from a database we built, and blend them into the template plate using Poisson blending. The location at which to paste these new features is determined by calculating areas on the surface with the lowest density of SIFT feature points, which we use as a proxy for surface homogeneity. We utilize areas of high homogeneity for blending in our features. The blending procedure that we use is Poisson blending, which match gradients between source and destination images, and works best in regions of high homogeneity. Finally, the image?s intensity values are altered so as to add a more fictional quality to it. Our algorithm takes advantage of many computational photography techniques and results in a variety of unique and exciting outputs. We hope that the new worlds will serve as great inspiration for science fiction settings.
Project webpage
Shane Bric and Kevin Paprocki
Portrait Obama-fication
We created an automated MATLAB program that turns a normal portrait into an Obama campaign style poster. The end result has a limited color base (4) and the colors are tiered in their appearance. We perform this transformation with the help of a bilateral filter, which blends like colors while preserving edges, and mapping colors based on resulting intensity. For ease of use, we also created a GUI to perform all the actions and change any values needed, such as filter options, intensity ranges, input colors and of course input images.
Project webpage
Ben Burchfiel and Aaron Cahn
Extending Eigenface based Facial Recognition with Retinex Theory
In this paper, we examine the performance and shortcomings of facial recognition using Eigenfaces. Specifically, we focus on the effects of lighting inconsistency on recognition success and examine one possible solution to the problem using an implementation of the Retinex algorithm to normalize lighting across an image. We found, given in register frontal parallel images, that with no lighting correction, faces in shadow were commonly misclassified. By using a spiral based Retinex approach, we were able to boost accuracy roughly 12 percent on our 150 face test group, showing that Retinex contrast reduction is a useful, and fast, extension for relighting in facial recognition approaches.
Project webpage
Travis Burleson and Ramsey Kropp
Automated Removal of Image-Blocking Objects
Several occupations require rapid entry into potentially hazardous closed spaces and would benefit from spatial information about the contents of the space. Recent research in computer vision and advances in computational speed have allowed the implementation of “structure from motion” methods to extract 3 dimensional structure from a video stream. These methods require a video stream that maintains an approximately constant orientation. A thrown video system suspended in a gimbal structure would provide this data, but the gimbal structure contains moving elements that would partially occlude the lens of the camera.
Our project aims to automatically identify a moving object with some characteristic trait (color or distinctive pattern), remove that object from any frames in which it is present, and use a combination of preceding and antecedent frame information and texture synthesis to fill in the voids left in those structures. We aim to use motion segmentation to assign motion to camera motion or object motion, remove those pixels which are associated with the moving object, and complete the final video sans the object. This is popularly known as “diminished reality,” as the video stream contains less information after processing. Only implemented thus far is the moving object removal and transfer of surrounding frame data.
Project webpage
Nathalie Cheng and Joe Kohlmann
Doodle Segment Querying and Colorization
Given a user's doodle (saved as a GIF with an indexed color table), the project recognizes the segments of the image drawn using the same color. It uses these color-grouped segments to query a database of images (as in Jacobs, et al.) based on shape alone (not color). The algorithm takes the resulting segment matches and colorizes them using the original doodle segment colors. Finally, it outputs the combined (and possibly blended) segments in a final image—a surreal recreation of the original doodle from real, color-shifted photographic data.
Imagine a doodle containing a green plain in the background, a blue sky, a yellow stick figure and a red car. Each segment of the image appears in a distinct color, so the algorithm extracts them, performs image matching queries with each, color corrects the matches, and blends the final image together from real images of a plain, a sky, a person, and a car. The photographic matches are all colorized to create a vivid realization of the original doodle.
Project webpage
Kevin Cisler, Brent Clark and Stephanie Scherer-Johnson
Camouflage Art
Motivated by the artwork of Liu Bolin, a famous Chinese artist whose works involve painting and blending himself into the background of his images, this project proposes to method replicate his artistic style. This project selects a face from a given photo, inserts this face into another background, and blends the face into the background image effectively camouflaging the person. Image matting in perceptual color space is implemented to extract the face initially. Edge filtering and Gaussian blurs are employed to add the face into the background image as if it were painted there.
Project webpage
David Delventhal and Xiaowei Wang
Hatching Rendering Based on a Single Photo
There exist many algorithms to create a non-photorealistic image from a photo. Many of these algorithms make the photo look like a painting or a cartoon. This project, however, gives the photo a hatching shading style. Existing techniques for hatching rendering either require the user to manually draw hatching or create a 3D model on which the algorithm rendering the hatching. This project focuses specifically on photos of architecture.
Project webpage
Andrew Dibble
Reading Devanagari
Here I build upon the algorithm for reading Devanagari advanced in the Ingalls and Ingalls paper "The Mahabharata: Stylistic Study, Computer Analysis, and Concordance." While they developed an application capable of accurately reading hand-written Devangari, I will use a single standard font, the Devanagari subset of Arial Unicode MS, although the algorithm presented here is capable of learning a wide range of Devanagari fonts. Also because of time constraints and because hundreds of templates are necessary to cover the full breadth of Devanagari, I will be dealing with a representative subset of the script. The primary contribution of the approach presented here is removal of the need to shift segmented characters over templates looking for the best match. The algorithm here simply involves resizing the segmented character to the size of the template.
Project webpage.
Samuel Farrell and Kenny Preston
Transforming Sketches into Vectorized Images
Our final project is an attempt to automate the process of digitizing analog forms of art. Unless a piece of work is initially created as digital medium, transforming the work into easily displayed and shared. Most artists import their sketches into professional image editing software such as Adobe Photoshop and manually edit and filter these images, sometimes even redrawing the whole thing using the original only as a guide. This work is painstaking and slow, and an automated program could truly speed this process up. In addition to the skill that created sketches takes, transforming them into a presentable digital format requires expertise using image editing software. The learning curve of these types of programs is often quite steep, and the thing an artist should be worrying about is tinkering around in software.
The resultant application has many uses, and would make transforming any physical images into digital copies a much easier process. Not limited to art, we believe that our application will have numerous uses in scientific fields as well. Researchers often compile notebooks full of sketches describing experiments and gathering data, which proves very difficult to save digitally. Scanning images results in quality loss, but is essentially the only way to digitize physical work. Using our application after scanning the data, we can not only reduce noise and data-loss, but we interpolate data and improve upon the original. Additionally, our design plays very well with text, and does a great job smoothing edges and intelligently filling in gaps.
Project webpage
Jean Forde and Charles Gorman
Defocus Magnification
Shortening the depth of field of an image is often desirable for creating artistic images and portraiture. Due to the limitations of many compact cameras, this can not always be done using hardware. We compare two different methods of adding blur to an image to simulate shortening the depth of field of an image. One technique builds on method which uses Elder-Zucker blur estimation to accurately recreate a blur kernel. The other method was a novel approach using edge detection to create an initial blur map and additional analysis of the initial blur map to create an optimized map. Both methods require minimal user input, and neither method requires accurate estimation of absolute image-object depth.
Project webpage
Seth Foss and Peggy Pan
Optical Character Recognition Preprocessing
In our project, we want to focus on optical character recognition (OCR), and modifying pre-existing code to enhance functionality. The available code works relatively well, but it turns out that the orientation matters when scanning, as it fails to correctly recognize characters when the text is tilted for more than a certain number of degrees. What we want to do is to make additions to the system so that it can recognize when there are characters in the image that are not oriented in the normal fashion. We believe that this is a useful application because in real life, more often than not, characters within images are not going to be aligned perfectly with the edges, so it would be nice if that issue can be detected and fixed automatically when the feature is not included in the OCR reader itself.
Project webpage
John Fritsche and Hans Werner
FACE-IT (Face Authoring Compiler Engine - Interactive Tool)
We present an image based technique of creating a novel face from a library of face parts. Our implementation uses cutting edge web technology to provide a simple user interface as a means of compiling a new face. The user selects from a set facial features broken down into eyes, noses, and mouths, placing them in any combination onto a base image. Our system uses a library of facial features, and re-renders a new image each time the user selects a different feature. This allows the interface to provide a real-time preview of results, and save the experimental faces at different stages if desired. Similar facial image manipulation mechanisms currently available differ from our system by keeping the main features of a subject's face together. What sets our idea apart is the freedom to separate the main human facial areas into a unitary feature, and combine them in any fashion. The new combination of parts is stitched together automatically entirely by the compiling engine; with no 3D information required or user assistance needed, other than the selection of a part. We have developed a system that can quickly produce images of human faces comprised of portions of multiple different photographs. Five distinct facial features are used to create a composite image: nose, mouth, left eye, right eye, and base facial structure These components are blended together to create what is hopefully a somewhat believable human face.
Project webpage
Keith Gould and James Stoffel
SKETCHIFY - Creation of Pencil Sketch like Videos and Pictures
The music video for A-ha's "Take On Me" features an animation style that resembles pencil strokes. These effects were probably achieved by drawing each frame individually. Our process will allow this effect to be achieved much faster by using software. Additionally, our method allows the use of source videos in addition to images. The process utilizes smoothing and gradient edge detection followed by some filter operations to simulate the characteristics of pencil strokes. Finally, more smoothing is applied for enhanced realism.
Project webpage
Daniel Greenheck
Procedural Generation of Normal Maps for Improved Realism in Rendering 3D Geometry
Normal mapping is a highly useful and inexpensive technique for adding realism to 3D models. Unfortunately, it often requires the assets of an artist which are not readily available for amateur studios or the hobbyist. Luckily, for a majority of the textures used in 3D modeling, procedural generation of normal maps is a viable option. By running the texture through a set of filters and tweaking a few settings appropriately, a modeler with no artistic experience is able to generate a professional looking normal map in a matter of seconds.
Project webpage
Kyle Halbach, Brian Hook and Mike Treffert
Photo Fusion
Photo Fusion is a (soon to be) web-based application destined to give the internet a one-stop shop for all their multi-photo needs. In its initial stages, we hope to combine photo mosaic and hockney joiner applications, with plans to possible add more features in the future (panoramas for example).
Project webpage
En Tao Ko and Gabe Stanek
Skin Color [Au]teration
We set out to try and convert a regular color image to an image with its contents colored gold. In order to achieve this end goal, we apply a series of filters to our input image. This method preserves the images texture values while resulting in an image that is colored gold.
Project webpage
Wes Miller
Expression Detection in Video
Expression detection is useful as a non-invasive method of lie detection and behavior prediction, as many facial expressions are involuntary. However, these facial expressions may be difficult to detect to the untrained eye. This program implements facial expression in video using optical strain as described by Godavarthy, and returns frames in which an expression is present, along with a processed image of the face showing the location(s) of strain, so that the expression can be more readily identified.
Project webpage
William Pelrine and Borui Wang
Display Color Calibration using Cameras
The parameters defining color output of a screen such as the gamma values for red, green and blue vary among manufacturers as well as among the monitors they make. To simplify and standardize this, a computer’s operating system defines the color preferences of the screen by querying an ICC profile based on a global standard defined by the International Color Consortium. We observed, however, that with the same ICC color profile loaded on two computers with different screens, the perceived color from the human eye could still be different due to the physical hardware differences of the screens. The color preference of a screen is crucial for users of software applications in the fields of graphic design and photography, where operating in true color is of utmost importance. Therefore, the need for screen color calibration arises as users require their work to look similar in different hardware context such as photo and video editors.
The current software solutions for screen color calibration not only require a professional background and an understanding of color spaces, but also require the purchasing of expensive hardware to work with the software such as a Spectracal CalMan, X-Rite EyeOne, or Colorvision Spider. In these examples, the cost of the solution ranges from $299 to $3,000. Our goal is to provide a simpler, cheaper solution that gives acceptable screen color calibration results for amateur users using only their own camera and a display. ur implementation includes the following. First, a program that decodes, encodes, and extracts information from the ICC binary file according to the tags defined by ICC. Second, an understanding of the relationship between the color space of RGB and the color space used by the ICC, as well as the correlation of brightness and gamma values of all color channels used in these two color spaces. Third, we need a Matlab program that measures image inputs from two screens and provides an offset in the color space used by ICC to map one screen's color to another. Finally, a Matlab program could be written that takes the difference of cameras into consideration. This is, however, merely an optimization and is not inherently necessary for our solution.
Project webpage
Scott Rick
VideoPhotoTourism - Using Video Input to Extend Phototourism
VideoPhototourism is a project which aims to extend phototourism applications to support video input. Phototourism applications such as Microsoft Photosynth and PhotoCity Game are able to create interesting 3D representations of popular landmarks and tourist locations by uploading, analyzing, and combing photos. My project is an application which makes incorporating videos into phototourism applications as simple as possible. Videophototourism is as simple as preprocessing videos using my MATLAB script to create video still images which are used as input to Photosynth. From one or more videos of tourist landmarks, a beautiful photosynth recreation of the tourist location is recreated online to share and enjoy.
Project webpage
Chad Sebranek and Mike Zwaska
Non-Rigid Point Matching: A Comparative Analysis
The topic of image matching on rigid objects, such as buildings, or statues, has been investigated in depth. The approaches used to handle the feature point matching share similar techniques. However, if one is to strip the assumption that the objects are rigid, then the cornerstone of all the approaches breaks. The task that remains is matching one point-set to another. However, due to lack of rigidity, the second point-set could have undergone some non-rigid transformation. Such a transformation is equivalent to deformation. Thus the task becomes non-rigid feature point matching. Solutions to this task are driven by many applications. The most novel is the use in keyframe animation, where 2 frames of an animation are drawn and the goal is to animation the motion between the frames. However, perhaps a more important application is in the medical field. Here it can be used to compare MRIs. Regardless of applications, there are several approaches to solving the problem, and this paper discusses and compares three state-of-the-art approaches.
Project webpage
Jacob Stoeffler
Photosaic
Photosaic is an automated photomosaic generator that uses a Java graphic user interface. The user provides the main photograph that the output image will resemble. Then they can choose either to provide the input images that will be tiled or they can choose to enter a keyword that will be used to find relevant images by using online image services, namely Flickr and Google Images. The user also chooses the tile size and the output image size, as well as a limit to the number of times each tile may be used. The program then generates the photomosaic and allows the user to save it where they wish.
Project webpage
Will Strinz
3D Scanning with Structure from Motion
Creating a three dimensional reconstruction of a real world object is an important tool for Compute Assisted Design (CAD) and rapid prototyping using 3D printers. Structure from Motion uses point correspondences between two dimensional images to solve for the location of the camera and the locations of the points in 3D space. This project uses algorithms in Matlab to turn images into printable 3D models with no calibration or interaction required beyond the initial image capture.
Project webpage
Allie Terrell
Group Photo Sharing on Facebook
Facebook has offered users a photo sharing application for the past several years. However, Facebook’s photo application is primitive at best, only allowing the most basic of organizational elements. Given that Facebook is one of the most popular ways to share information with one’s social network, it seems a logical next step that Facebook allows users to share photos as well.
This paper explores photo application use trends on Facebook, focusing on photo sharing. The data collected motivates the design of three new ways to augment Facebook to allow for more fluid photo sharing amongst friends and family members. Privacy has always been a popular point of contention in regards to Facebook. The issue of privacy in regards to all three of these techniques is also discussed.
Project webpage

CS 534 | Department of Computer Sciences | University of Wisconsin - Madison