结合光流法和卡尔曼滤波的视频稳像算法

计算机工程与科学

结合光流法和卡尔曼滤波的视频稳像算法

熊炜1,2，王传胜1，李利荣1，刘敏1，曾春艳1

（1.湖北工业大学电气与电子工程学院，湖北武汉 430068；

2.美国南卡罗来纳大学计算机科学与工程系，南卡哥伦比亚 29201）

收稿日期:2019-09-26 修回日期:2019-10-20 出版日期:2020-03-25 发布日期:2020-03-25
基金资助:
国家留学基金(201808420418)；国家自然科学基金(61571182,61601177)；湖北省自然科学基金(2019CFB530)

Video stabilization algorithm based on

optical flow method and Kalman filtering

XIONG Wei1,2,WANG Chuan-sheng1,LI Li-rong1,LIU Min1,ZENG Chun-yan1

#br#

(1.School of Electrical and Electronic Engineering,Hubei University of Technology,Wuhan 430068,China;

2.Department of Computer Science and Engineering,University of South Carolina,Columbia,SC 29201,USA)

Received:2019-09-26 Revised:2019-10-20 Online:2020-03-25 Published:2020-03-25

摘要/Abstract

摘要：

针对手机拍摄过程中产生的视频抖动问题，提出了一种基于光流法和卡尔曼滤波的视频稳像算法。首先通过光流法预稳定抖动视频，对其生成的预稳定视频帧进行Shi-Tomasi角点检测，并采用LK算法跟踪角点，再利用RANSAC算法估计相邻帧间的仿射变换矩阵，由此计算得出原始相机路径；然后通过卡尔曼滤波器优化平滑相机路径，得到平滑相机路径；最后由原始相机路径与平滑路径的关系，计算相邻帧间的补偿矩阵，再利用补偿矩阵对视频帧逐一进行几何变换，由此得到稳定的视频输出。实验表明，该算法在处理6大类抖动视频时均有较好的效果，其中稳像后视频的PSNR值相比原始视频的PSNR值约提升了6.631 dB，视频帧间的结构相似性SSIM约提升了40%，平均曲率值约提升了8.3%。

关键词: 视频稳像, 预稳定, Shi-Tomasi角点, LK算法, RANSAC算法, 卡尔曼滤波

Abstract:

Aiming at the video jitter problem caused by mobile phone shooting, a video stabilization algorithm based on optical flow method and Kalman filter is proposed. Firstly, the optical video method is used to pre-stabilize the dithered video, and Shi-Tomasi corner detection is performed on the pre-stabilized video frame. The LK algorithm is used to track the corner points, and then the RANSAC algorithm is used to estimate the affine transformation matrix between adjacent frames, so as to obtain the original camera path. Secondly, the smoothed camera path is optimized by the Kalman filter to obtain a smooth camera path. Finally, the relationship between the original camera path and the smooth path is used to calculate the compensation matrix between adjacent frames, and then the compensation matrix is used to geometrically transform the frames one by one, resulting in a stable video output. Experiments show that the proposed algorithm has good effects in processing six types of jitter videos. The PSNR value after image stabilization is increased by 6.631 dB compared with the original video, and the structural similarity (SSIM) between video frames is increased by 40%. The average curvature value is increased by about 8.3%.

Key words: video stabilization, pre-stability, Shi-Tomasi corner point, LK algorithm, RANSAC algorithm, Kalman filter

熊炜1,2，王传胜1，李利荣1，刘敏1，曾春艳1. 结合光流法和卡尔曼滤波的视频稳像算法[J]. 计算机工程与科学.

XIONG Wei1,2,WANG Chuan-sheng1,LI Li-rong1,LIU Min1,ZENG Chun-yan1.

Video stabilization algorithm based on

optical flow method and Kalman filtering

[J]. Computer Engineering & Science.

[1]	熊炜1,2，王传胜1，管来福1，童磊1，刘敏1，曾春艳1. 基于特征跟踪和网格路径运动的视频稳像算法[J]. 计算机工程与科学, 2020, 42(05): 843-850.
[2]	宋建材1,3,候春萍1,薛桂香2. 基于扩展卡尔曼滤波的时钟模型辅助定位算法[J]. 计算机工程与科学, 2018, 40(04): 740-744.
[3]	刘召1,2，宋立滨2，于涛2，郭凯2，王增喜2，耿美晓2. 基于IMU的主动伴舞机器人人机协作控制算法[J]. 计算机工程与科学, 2018, 40(01): 128-132.
[4]	施啸天1，张宇2，方舟1. 基于前后双摄像头的鲁棒视觉里程计算法[J]. 计算机工程与科学, 2017, 39(10): 1862-1869.
[5]	秦煜1，吴静静1,2，安伟1,2，张美娟3. 基于融合特征的SURF配准算法[J]. 计算机工程与科学, 2017, 39(10): 1890-1895.
[6]	张昆1，陶建锋2，贺思三2. 基于UKF和优化组合策略的改进粒子滤波算法[J]. 计算机工程与科学, 2017, 39(08): 1483-1488.
[7]	秦煜1，吴静静1,2，安伟1,2. 基于RANSAC的激光网格标记图像特征提取[J]. 计算机工程与科学, 2017, 39(08): 1495-1501.
[8]	王祎1，杏建军1，于洋1，郑黎明1，赵延鹏1，刘琛2. 编队卫星相对位置测量信息自主确定月球引力场的方法[J]. J4, 2016, 38(01): 125-130.
[9]	安婷，贺一民，张志毅. 改进的双向SIFT特征匹配算法[J]. J4, 2016, 38(01): 138-143.
[10]	何晓亮1,2，宋威1，梁久祯1. 基于资源分配网络和语义特征选取的文本分类[J]. J4, 2014, 36(02): 340-346.
[11]	孙雷1,2，李冬1，易东云1. 基于迭代UKF的主动段弹道跟踪算法研究[J]. J4, 2014, 36(01): 121-125.
[12]	董立红. 基于卡尔曼滤波的刮板输送机张力控制系统[J]. J4, 2013, 35(4): 186-190.
[13]	蒋学斌. 微固态航姿系统的仿真与设计[J]. J4, 2012, 34(9): 193-196.
[14]	余东，黄文明，温佩芝，宁如花，黄锦芳. 基于卡尔曼滤波改进的精子图像序列分割方法[J]. J4, 2012, 34(7): 104-108.
[15]	吴飞. 一种具有在线参数调整功能的Kalman滤波及其应用[J]. J4, 2012, 34(6): 93-96.