• 中国计算机学会会刊
  • 中国科技核心期刊
  • 中文核心期刊

计算机工程与科学

• 图形与图像 • 上一篇    下一篇

基于特征跟踪和网格路径运动的视频稳像算法

熊炜1,2,王传胜1,管来福1,童磊1,刘敏1,曾春艳1   

  1. (1.湖北工业大学电气与电子工程学院,湖北 武汉 430068;
    2.美国南卡罗来纳大学计算机科学与工程系,南卡 哥伦比亚 29201)
  • 收稿日期:2019-06-20 修回日期:2019-09-11 出版日期:2020-05-25 发布日期:2020-05-25
  • 基金资助:

    国家留学基金(201808420418);国家自然科学基金(61571182);湖北省自然科学基金(2019CFB530)

A video stabilization algorithm based
on feature tracking and mesh path motion

XIONG Wei1,2,WANG Chuan-sheng1,GUAN Lai-fu1,TONG Lei1,LIU Min1,ZENG Chun-yan1   

  1. (1.School of Electrical and Electronic Engineering,Hubei University of Technology,Wuhan 430068,China;
    2.Department of Computer Science and Engineering,University of South Carolina,Columbia,SC 29201,USA)
  • Received:2019-06-20 Revised:2019-09-11 Online:2020-05-25 Published:2020-05-25

摘要:

针对手持移动设备拍摄的抖动视频问题,提出了一种基于特征跟踪和网格路径运动的视频稳像算法。通过SIFT算法提取视频帧的特征点,采用KLT算法追踪特征点,利用RANSAC算法估计相邻帧间的仿射变换矩阵,将视频帧划分为均匀的网格,计算视频的运动轨迹,再通过极小化能量函数优化平滑多条网格路径。最后由原相机路径与平滑相机路径的关系,计算相邻帧间的补偿矩阵,利用补偿矩阵对每一帧进行几何变换,从而得到稳定的视频。实验表明,该算法在手持移动设备拍摄的抖动视频中有较好的结果,其中稳像后视频的PSNR平均值相比原抖动视频PSNR值大约提升了11.2 dB。与捆绑相机路径方法相比约提升了2.3 dB。图像间的结构相似性SSIM平均值大约提升了59%,与捆绑相机路径方法相比约提升了3.3%。

 

关键词: 视频稳像, SIFT算法, KLT追踪, RANSAC算法, PSNR, SSIM

Abstract:

 

 


A video stabilization algorithm based on feature tracking and mesh path motion is proposed to solve the jitter video issues for handheld mobile devices. The algorithm uses SIFT algorithm to extract the feature points of video frames, uses KLT algorithm to track the feature points, uses RANSAC algorithm to estimate the affine transformation matrix between adjacent frames, divides the video frames into uniform grids, calculates motion trajectories of the video, and then optimizes the smoothing of multiple mesh paths by minimizing the energy function. Finally, the compensation matrix between adjacent frames is calculated by the relationship between the original camera path and the smoothed camera path, and then each frame is geometrically transformed by the compensation matrix to obtain a stable video. Experiments show that the proposed algorithm has good results for the jitter video captured by handheld mobile devices. The average PSNR after image stabilization is approximately 11.2 dB higher than that of the original jitter video, and is approximately 2.3 dB higher than the bundled camera path method. The average structural similarity (SSIM) between images is increased by approximately 59%, and is approximately 3.3% higher than the bundled camera path method.
 

Key words: video stabilization, SIFT algorithm, KLT tracking, RANSAC algorithm, PSNR, SSIM