Input View 1
Synthesized View
Input View 2
|
View Interpolation |
View interpolation is the process of creating a sequence of synthetic images that, taken together, represent a smooth transition from one view of a scene to another view. Two themes have dominated my work in this area: (1) utilizing only two original views of the scene to generate the sequence and (2) allowing for scenes that contain motion (called dynamic scenes).
Typically, view interpolation algorithms also allow for extrapolation (i.e., continuing to generate synthetic images beyond the end of the interpolation sequence); this is true of all of my algorithms.
|
View Interpolation for Dynamic Scenes |
My first view-interpolation research extended Steve Seitz's view morphing technique to work with certain kinds of dynamic scenes; the work was called dynamic view morphing (see Russell A. Manning and Charles R. Dyer, Interpolating view and scene motion by dynamic view morphing, Proc. Computer Vision and Pattern Recognition, Fort Collins, Colorado, June 1999, pages I:388-394).
The Problem: Given two views of a dynamic scene, taken by different cameras in different locations at different times, create a sequence of views showing a smooth transition between the two camera views that also shows the dynamic scene changing smoothly.
Example: The left and right images below are photographs taken in our old Vision Lab. The pictures were taken from two different positions using a different focal length each time. Everything in the scene was static EXCEPT for the blue box in the foreground, which underwent a translation between when the views were captured. Our goal is to create a series of images showing what happened during the missing interval of time. In particular, we want to interpolate both the viewing position and any motion that occured in the scene.
From the original reference views alone it would be impossible to state exactly how the box moved during the missing time. However, a reasonable guess is that the box slid in a straight line; in this case it would move parallel to the lines in the floor. Our technique makes this assumption of "straight-line" motion when creating dynamic view interpolation sequences. Results are given below as an "AVI" movie, originally shown at CVPR99. Note that the technique is automatic provided the algorithm is given two things: (1) sufficient point correspondences between the two original reference views, and (2) knowledge of the boundaries of each moving object (i.e., each object is placed on its own "layer"). In our work, both (1) and (2) are determined manually. Furthermore, some effort has been expended to ensure that (1) and (2) are chosen correctly so as to produce nice-looking interpolation sequences.
|
Input View 1 |
Synthesized View |
Input View 2 |
|---|---|---|
Here is another example; this was from VSAM98:
|
Input View 1 |
Synthesized View |
Input View 2 |
|---|---|---|
|
Interpolation of Augmented Scenes |
Because of the robustness of my approach, it is possible to "paste" a synthetic object onto the two reference views so that it looks approximately correct for the scene and then to interpolate between the augmented views. In the example below, a synthetic truck object is added to look like it rests on the table top in both reference views. The truck changes position between the reference views; the interpolation sequence shows the motion of the synthetic object.
|
Input View 1 |
Synthesized View |
Input View 2 |
|---|---|---|
Entire sequence is available as a movie: [0.1M mpeg] [1.5M SGI Cosmo] [6.3M QuickTime]
|
View Interpolation for Environment Maps |
As originally published, the view morphing algorithm did not allow for camera motion into or out of the viewing window (i.e., the epipole between the two views could not be contained in either view). Because dynamic scenes would often contain objects moving towards or away from the camera, and in general, because allowing camera motion towards or away from the view was desirable, I created the environment map morphing technique. Put simply, an environment map is a panoramic or wide-angle view; in the extreme, an enviroment map can be the entire sphere around the camera. Environment map morphing makes it possible to perform view interpolation between two arbitrary, uncalibrated environment maps while preserving the scanline property.
|
First Frame |
Synthesized Middle Frame |
Last Frame |
|---|---|---|
Entire sequence is available as a movie: [0.5M mpeg] [4.8M SGI Cosmo] [13M QuickTime]
|
Russell Manning / rmanning@cs.wisc.edu / last modified 01/22/01 |
