I am currently investigating applying image-based rendering to scenes containing moving objects, and trying in general to extend the capabilities image-based rendering. The following examples highlight progress to date.
The Problem: Given two views of a dynamic scene, taken by different cameras in different locations at different times, create a sequence of views showing a smooth transition between the two camera views that also shows the dynamic scene changing smoothly.
Example: The left and right images below are photographs of the desk in my office, taken from two different positions and with two different focal lengths. All of the objects in the scene (the books, papers, computer monitor, etc.) remained stationary EXCEPT for the orange-colored box in the foreground. The box is photographed in two different positions, although in both positions it is flush with the edge of the table. From the original reference views it is impossible to know exactly how the box moved during the missing time. However, a reasonable guess is that the box slid in a straight line along the edge of the table. If the object had been a car driving across a bridge, then it would have been crucial to portray the car traveling in a straight line.
| Input View 1 | Synthesized View | Input View 2 |
|---|---|---|
|
|
|
|
| Input View 1 | Synthesized View | Input View 2 |
|---|---|---|
|
|
|
| First Frame | Synthesized Middle Frame | Last Frame |
|---|---|---|
|
|
|