Mid-Term Report

Overview

As expected in our project schedule, we have finished 1-dimension and 2-dimension seam carving. We applied a filter and get the edge-based energy map for the input, using dynamic programming to find the optimal seam and removed it once per iteration. The two directional removal of seams are implemented with similar idea, we compress the longer dimension first for now while transferring python to Cython code, after that we will have a table trying reasonable orders and returns global/local optimum. We write the codes in Python. In some cases such as calculating the optimal seam carving, pure Python code is slow and cause too much time, due to the slowness of python when iterating a loop, we embed C code to fasten it.

Progress

Edge-base measure for input image.

We first detect the edge in order to have a better performance making use of the "operator" mentioned in the paper who calculates the importance of pixels

Implementation of dynamic programming code to find optimal seam for one direction

Given an energy function e, we can define the cost of a seam as the sum of energy value of all the points on the seam. We use dynamic programming method to find the optimal seam who has the lowest energy value and thus have minimal effect on the original image. while traversing the image from the second row to the last row. For every points on every row, set the cumulative energy value equals to the current energy value plus the minimum cumulative energy value in the connected points in the last row.

Energy Function

We choose the square function who computes the difference between immediate neighbors as our energy for now. Since numpy support ndarray computation, it does not slow our process down very much.

Optimal Order

Resize along both horizontal and vertical direction with optimal order. To find the optimal order of removal, a dynamic programming table is built to store the order of removal for all possible size. Upon the request of a given size, which m columns and n row to be removed, the order of removal is determined from the dp table and the total computing time is O(m*n)

Result

Origin

Fig.1. Original Image

Removing Vertical seams

Fig.2. Left:Result of removing 200 vertical seams. Right: the seams to be removed

Adding vertical seams

Fig.3. Left:Result of adding 200 vertical seams. Right: the seams to be added

Removing horizontal seams

Fig.5. Left:Result of removeing 200 horizontal seams. Right:the seams to be removed

Adding horizontal seams

Fig.5. Left: Result of adding 200 horizontal seams. Right: the seams to be added

Removing both directions

Fig.6. Result of removing 100 horizontal seams and 100 vertical seams

Challeges

  • First we used node.js to implement the algorithm, unfortunately we found that node.js bad support for various image files.
  • After changing to python, we found that the loop takes too much time because we need to do the DP and get the best seam from the whole image in each iteration, thus we embed python in it to make it way more faster. However, we are still in process of transferring code from python to C and that may takes much time because we are unfamiliar with Cython and the document is not well.
  • Our energy function is not ideal and we are trying to make it better by pre-process the image using the common filters to detect edges.
  • In two dimensional resizing, because we can do seam insertion/removing in random order, it is exponential to the number of seams we want to remove. Thus, we now have a fixed order and plan to try reasonable number of orders and returns global/local optimum before taking too much time.
  • The content amplification and elimination will be implemented soon on the base of the 1d and 2d algorithm we have.