基于特征跟踪和网格路径运动的视频稳像算法

计算机工程与科学

基于特征跟踪和网格路径运动的视频稳像算法

熊炜1,2，王传胜1，管来福1，童磊1，刘敏1，曾春艳1

（1.湖北工业大学电气与电子工程学院,湖北武汉 430068;

2.美国南卡罗来纳大学计算机科学与工程系，南卡哥伦比亚 29201）

收稿日期:2019-06-20 修回日期:2019-09-11 出版日期:2020-05-25 发布日期:2020-05-25
基金资助:
国家留学基金(201808420418)；国家自然科学基金(61571182);湖北省自然科学基金（2019CFB530）

A video stabilization algorithm based

on feature tracking and mesh path motion

XIONG Wei1,2,WANG Chuan-sheng1,GUAN Lai-fu1,TONG Lei1,LIU Min1,ZENG Chun-yan1

（1.School of Electrical and Electronic Engineering,Hubei University of Technology,Wuhan 430068,China;

2.Department of Computer Science and Engineering,University of South Carolina,Columbia,SC 29201,USA）

Received:2019-06-20 Revised:2019-09-11 Online:2020-05-25 Published:2020-05-25

摘要/Abstract

摘要：

针对手持移动设备拍摄的抖动视频问题，提出了一种基于特征跟踪和网格路径运动的视频稳像算法。通过SIFT算法提取视频帧的特征点，采用KLT算法追踪特征点，利用RANSAC算法估计相邻帧间的仿射变换矩阵，将视频帧划分为均匀的网格，计算视频的运动轨迹，再通过极小化能量函数优化平滑多条网格路径。最后由原相机路径与平滑相机路径的关系，计算相邻帧间的补偿矩阵，利用补偿矩阵对每一帧进行几何变换，从而得到稳定的视频。实验表明，该算法在手持移动设备拍摄的抖动视频中有较好的结果，其中稳像后视频的PSNR平均值相比原抖动视频PSNR值大约提升了11.2 dB。与捆绑相机路径方法相比约提升了2.3 dB。图像间的结构相似性SSIM平均值大约提升了59%，与捆绑相机路径方法相比约提升了3.3%。

关键词: 视频稳像, SIFT算法, KLT追踪, RANSAC算法, PSNR, SSIM

Abstract:

A video stabilization algorithm based on feature tracking and mesh path motion is proposed to solve the jitter video issues for handheld mobile devices. The algorithm uses SIFT algorithm to extract the feature points of video frames, uses KLT algorithm to track the feature points, uses RANSAC algorithm to estimate the affine transformation matrix between adjacent frames, divides the video frames into uniform grids, calculates motion trajectories of the video, and then optimizes the smoothing of multiple mesh paths by minimizing the energy function. Finally, the compensation matrix between adjacent frames is calculated by the relationship between the original camera path and the smoothed camera path, and then each frame is geometrically transformed by the compensation matrix to obtain a stable video. Experiments show that the proposed algorithm has good results for the jitter video captured by handheld mobile devices. The average PSNR after image stabilization is approximately 11.2 dB higher than that of the original jitter video, and is approximately 2.3 dB higher than the bundled camera path method. The average structural similarity (SSIM) between images is increased by approximately 59%, and is approximately 3.3% higher than the bundled camera path method.

Key words: video stabilization, SIFT algorithm, KLT tracking, RANSAC algorithm, PSNR, SSIM

熊炜1,2，王传胜1，管来福1，童磊1，刘敏1，曾春艳1. 基于特征跟踪和网格路径运动的视频稳像算法[J]. 计算机工程与科学.

XIONG Wei1,2,WANG Chuan-sheng1,GUAN Lai-fu1,TONG Lei1,LIU Min1,ZENG Chun-yan1.

A video stabilization algorithm based

on feature tracking and mesh path motion

[J]. Computer Engineering & Science.

[1]	熊炜1,2，王传胜1，李利荣1，刘敏1，曾春艳1. 结合光流法和卡尔曼滤波的视频稳像算法[J]. 计算机工程与科学, 2020, 42(03): 493-499.
[2]	刘艳萍1，崔彤1，周长兵2，李小翠2，刘甜1. 基于改进混合高斯模型和图形句柄的异常车辆检测[J]. 计算机工程与科学, 2020, 42(02): 266-272.
[3]	秦煜1，吴静静1,2，安伟1,2，张美娟3. 基于融合特征的SURF配准算法[J]. 计算机工程与科学, 2017, 39(10): 1890-1895.
[4]	苏培峰1，黄世奇2，王艺婷1，刘代志1. 具有SIFT描述的多尺度角点图像配准[J]. 计算机工程与科学, 2017, 39(09): 1700-1705.
[5]	秦煜1，吴静静1,2，安伟1,2. 基于RANSAC的激光网格标记图像特征提取[J]. 计算机工程与科学, 2017, 39(08): 1495-1501.
[6]	兰红,洪玉欢,高晓林. 面向全景拼接的图像配准技术研究及应用[J]. J4, 2016, 38(02): 317-324.
[7]	安婷，贺一民，张志毅. 改进的双向SIFT特征匹配算法[J]. J4, 2016, 38(01): 138-143.
[8]	常方媛1，冯志勇1，徐超2. 基于SLIC区域分割的三维地形重建算法[J]. J4, 2015, 37(09): 1718-1723.
[9]	刘婧,王威，李骥，杨蔚蔚. 基于对偶树复小波变换的模糊图像质量评价[J]. J4, 2015, 37(08): 1573-1578.
[10]	叶红[1] 安东升[2]. 基于小波变换的卫星云图压缩方法[J]. J4, 2004, 26(10): 60-62.
[11]	刘杰康克军等. 小目标图象压缩的研究[J]. J4, 2001, 23(1): 44-46.