Synthetic Examples: Astronaut Sequence
The astronaut sequence, played back at 10 frames per second. The sequence has a variable exposure time with noise added to simulate adjustments to the gain. Click on the video to start playing.
The piecewise cubic spline used to generate the astronaut sequence. The foreground moves along the spline from right to left. Heavier segments of the curve indicate the exposures used to simulate motion blur.
- Flow result from frame 4 to 5
- Flow result from frame 10 to 9
- Concatenated flow result from frame 1 to 5
Flow fields are visualized using hue to represent angle and saturation to represent magnitude, as shown in the image below.
Pairwise flows
Frame 4 to frame 5
In the first example, the source (frame 5) is sharp and the target (frame 4) is blurred. The flow visualizations are shown below. The “ground-truth” flow is generated using the centers of the exposures.
Please move your mouse cursor over the small images to toggle between the results.
| Target | Source | Baseline | Our method | Ground truth |
|---|---|---|---|---|
|
|
|
|
|
Frame 4 (target) |
||||
Our flow is much more consisent and better resembles the ground-truth flow than that produced by the baseline method.
Warping results from the source image (frame 5) to the target image (frame 4) using estimated flows are shown below.
| Source | Baseline | Our method | Target |
|---|---|---|---|
|
|
|
|
|
|||
| Ground-truth target |
|||
Input source (frame 5) |
|||
Our method better preserves the sharpness of the source while warping it to the location of the target, because the baseline method attempts to warp the sharp source image to match the blurry appearance of the target.
Frame 10 to frame 9
In this example, the source (frame 9) is blurred and the target (frame 10) is sharp.
| Target | Source | Baseline | Our method | Ground truth | ||||
|---|---|---|---|---|---|---|---|---|
|
|
|
|
|
||||
Frame 10 (target) |
||||||||
Again, our flow is much more consisent and better resembles the ground-truth flow.
Warping results from the source image (frame 9) to the target image (frame 10) using estimated flows are shown below.
| Source | Baseline | Our method | Target | |
|---|---|---|---|---|
| Warping using input frames |
|
|
|
|
| Warping using “ground-truth” frames |
|
|
|
|
Input source (frame 9) |
||||
In the first row, the baseline method tries to warp the blurred source frame to match the sharpness of the target frame. Therefore, the baseline warping result is a distorted version of the source. To better visualize the distortion, in the second row, we use the estimated flow to warp the corresponding ground-truth frames (no blur, no noise). Our method better preserves the structure of the source.
Concatenated flows
To better see the differences between the two methods, we concatenate pairwise flows to get flows between non-consecutive frames. This allows us to warp between two sharp frames that have one or more blurred frames separating them. Ideally, the warping result should match the sharp target frame. No additional refinement or processing is performed on the concatenated flow fields.
Frame 1 to frame 5
In this example, the source (frame 5) and the target (frame 1) are both sharp. Multiple blurred frames separate the source and target.
| Source | Baseline | Our method | Target | |
|---|---|---|---|---|
|
|
|
|
|
Source (frame 5) |
||||
Our warping matches the target frame much better than the baseline warping, which severely distorts most of the foreground (e.g., the red stripes on the legs and the instruments on the torso). This can be seen by toggling between the two warping results and the target. Note that the distortion of the background in both results is caused by occlusion where the pixels in the target have no corresponding pixels in the source.