Rendering Images Using Objects as Primitives

Lorenzo De Carli

Introduction

Throughout the centuries, many artists created stunning visual effects by rendering well-known physical objects using combinations of other objects. The first to do so was probably the XVI-century Italian painter Giuseppe Arcimboldo. The technique is also popular among modern artists such as Octavio Ocampo. This webpage describes an algorithm which "emulates" the technique by rendering a source picture using images from an object library. The algorithm is quite simple, runs fast and it is inspired by related works such as Texture Transfer and Photomosaics. Example results and a Matlab implementation (based on the MIT CSAIL LabelMe library) are also provided.

The Method

Since no such technique is already present in the image processing literature, I had to come up with a name - Object-based Rendering is short, easy to remember and sounds pretty cool :-). This technique is similar to Photomosaics and Texture Transfer (see Links section), in the sense that small picture of objects are used as "macropixels" to render the initial image. Such small pictures of objects are picked from an image library that the user must provide. The algorithm works by computing all the possible matchings of the objects in the library with the source image - computing the matching means to compute the pixel-by-pixel difference between the colors of the two images. The smaller is the difference, the closer the object is to a specific region of the image. Once this process is done, the closest matchings are chosen and used to generate the result.

More details on the functioning of the algorithm can be found at this page.

Download: [Object-based Rendering Matlab implementation].

Results

Object-based rendering requires an object library to work. For testing purposes, I created two libraries (using the LabelMe image collection as a starting point), which I called Fruit and Fish. You can see the objects included in each library here. Once a library has been created, the algorithm can be run in two different modes:

Full image rendering
In this mode, the whole image is rendered in a single step. This approach leads to somewhat confused results, because the technique is not able to closely follow the image features. A sample result obtained in this way can be found here.

Multiple regions rendering
In this mode, the user divides the image into multiple regions, and decides which objects to use to render each region. The increased user interaction leads to much better results. Example results can be found at this page.

Note

The code was originally created in Spring 2009 as a course project for the Computational Photography class (CS638) at UW-Madison, held by Professor Chuck Dyer.

Back to my home page

Last update: November 9 2009.