Screw-Transform Manifolds for Camera Self Calibration
R. A. Manning, Ph.D. Dissertation, University of Wisconsin - Madison, September 2003.

Abstract

This dissertation concerns the mathematical theory of screw-transform manifolds and their use in camera self calibration. A camera's calibration is the function that maps 3D scene points to 2D image points, e.g., in photographs taken by the camera. Between every two photographs taken from different positions there exists a pairwise constraint called the "fundamental matrix" which can be computed directly from the images. When the two photographs are captured by the same camera, the fundamental matrix induces a surface in calibration space called a "screw-transform manifold." This manifold represents every possible internal calibration for the camera. By acquiring several different pairwise fundamental matrices, it is possible to compute several different screw-transform manifolds; however, the internal calibration of the camera must be a member of each such manifold and hence, by finding the intersection point of all the manifolds, the camera's calibration can be determined. The process of determining calibration directly from images taken by a camera is called "self calibration."

The contributions of this dissertation include the theory of screw-transform manifolds and three original algorithms for determining the mutual intersection points of a collection of manifolds. While many papers have been written on self calibration, almost all previous methods posed their solutions as the global minima of an objective function. However, performing global optimization is problematic; it is easy to locate a local minimum without finding the global minimum, and in some cases the attraction basin of the global minimum is so small that the algorithm must essentially "guess" the solution in order to find it. One of the new approaches created as part of this dissertation, called STM-SURFIT, avoids optimization altogether but nevertheless locates all global minima (i.e., solutions to the internal calibration problem) in a single pass, running in under 1 second on a modern home computer. This general approach to avoiding the difficulties of optimization may have wider applicability beyond camera calibration.

A tutorial on multiview geometry that assumes only knowledge of linear algebra is included to provide the necessary mathematical background. The related history and previous work on self calibration and image-based rendering is also presented. As part of the theory of screw-transform manifolds, a theorem is introduced that partitions monocular view pairs into six categories based on the underlying screw displacement of the camera and provides a simple test for determining category. In addition, some methods for self calibration and image-based rendering for dynamic scenes are presented. These latter image-based rendering techniques do not require camera calibration but are limited in applicability, thus adding to the growing body of evidence that camera calibration is necessary for useful image-based rendering.