Discovering panoramas in web videos

Feng Liu1, Yuhen-Hu1,2 and Michael Gleicher1

1Computer Sciences Department, University of Wisconsin-Madison
Department of Electrical and Computer Engineering, University of Wisconsin-Madison

While panorama stitching methods have been successful given proper source images, providing these source images still remains a burden. In this paper, we present a method to discover panoramic source images within widely available existing videos, such as videos on YouTube. The challenge comes from the fact that many of these videos are recorded casually, not intended for stitching panoramas. Our method aims to find segments within a video that are qualified as panorama sources. Specifically, we determine a video segment a valid panorama source according to the following three criteria. First, its camera motion should cover a wide field-of-view of the scene. Second, its frames should be "mosaicable'', which states that the inter-frame motion should observe the underlying conditions for stitching a panorama. Third, its frames should have good image quality. Based on these criteria, we formulate discovering panoramas in a video as an optimization problem that aims to find an optimal set of video segments as panorama sources. After discovering these panorama sources, we synthesize different types of panorama images based on the source characteristics. For example, we stitch scene panorama or activity synopsis based on the video dynamic analysis. The scene panorama conveys a wide field-of-view of the scene and the activity synopsis conveys the video dynamics. Our experiments of querying panoramas from YouTube confirm the feasibility of using casually recorded videos as panorama sources and demonstrate the effectiveness of our method.

Working example. A user makes a query of  "West Lake, Hangzhou" to YouTube, and feeds retrieved video clips into our system. Our system selects useful frames from the given videos and synthesizes panoramas using the selected source frames.

Feng Liu, Yu-hen Hu and Michael Gleicher. Discovering panoramas in web videos. 
ACM Multimedia 2008, Vancouver, Canada, October 2008. pp. 329-338. PDF
We made queries to YouTube Travel and Events category, and ran our algorithm on the top 10 retrieved videos. Below are some discovered panoramas as well as corresponding source videos. 
Note: Please click each image in the following pages to view its full-size version. 
Query: Delicate Arches National Park
Query: Notre Dame, Paris
Query: West Lake, Hangzhou
Query: Lake Louise, Canada
Query: Vancouver Beach
Query: Hong Kong Skyline