Sum of difference for static seams.

Seam searching.

Step by step results.


Exploring Spatial and Temporal Coherence to Strengthen Seam Carving in Video Retargeting




Number of various display screens and mobile devices has increased significantly. Unfortunately, showing a fixed size picture or video is often limited by the aspect ratio of different displays. In recent years, many content-aware retargeting techniques have been proposed. Among them, Seam carving is a novel and efficient method, but it may distort the object’s structure.

For enlarging an image, we tend to make it larger and undistorted by first magnifying the image, and shrink it to the target size using Seam carving. Thus, in this thesis, we focus on shrinking. For spatial coherence, we emphasize the object shape and protect significant content. We also combine Seam carving and Scaling operator, trying to avoid the bad results due to content distortion. Moreover, we extend our method to video retargeting, which formerly caused the jittery artifacts without exploring temporal information. We classify the videos into those taken by the static camera setup and the others by the moving camera setup. Then we explore temporal coherence to decrease the jittery artifacts. Finally, the experimental results demonstrate our approach can raise the quality in video retargeting.


SUMMARY (中文總結):






1.Spatial coherence cost

  我們提出了Compound energy function,分別有Forward energySaliency mapCanny detector,藉由各方法的好處來降低內容扭曲以及保護重要物件的效果;為了防止使用者需等待過多時間來執行seam carving,又得不到預期的好結果,我們在每個刪除seam的階段都會執行品質的評估,若是相似度變低了,我們將使用scale 運算子來縮小到目標大小。


2.Temporal coherence cost

  在影片的分類上,我們大致分成兩種類型(static cameramoving camera)來做處理。在static camera的影片,我們將使用sum of difference來記錄物件移動的軌跡,並加入第一個frameenergy來計算出static seams,之後的frames就可以利用查表的方式來直接移除seams,因此可以減少計算的時間和降低影片抖動的現象;在moving camera的影片,我們使用收尋seam的方法,找到當前frame與上一個frameseam最相似的seam,如果收尋不到我們會額外加入temporal coherence cost,來重新計算出一條最接近的seam,最後,我們就能減少抖動的現象,並得到好的結果。



§  Video results.

§    結果展示