• 中国计算机学会会刊
  • 中国科技核心期刊
  • 中文核心期刊

计算机工程与科学

• 图形与图像 • 上一篇    下一篇

结合光流法和卡尔曼滤波的视频稳像算法

熊炜1,2,王传胜1,李利荣1,刘敏1,曾春艳1   

  1. (1.湖北工业大学电气与电子工程学院,湖北 武汉 430068;
    2.美国南卡罗来纳大学计算机科学与工程系,南卡 哥伦比亚 29201)
     
  • 收稿日期:2019-09-26 修回日期:2019-10-20 出版日期:2020-03-25 发布日期:2020-03-25
  • 基金资助:

    国家留学基金(201808420418);国家自然科学基金(61571182,61601177);湖北省自然科学基金(2019CFB530)

Video stabilization algorithm based on
optical flow method and Kalman filtering

XIONG Wei1,2,WANG Chuan-sheng1,LI Li-rong1,LIU Min1,ZENG Chun-yan1
#br#  
  

  1. (1.School of Electrical and Electronic Engineering,Hubei University of Technology,Wuhan 430068,China;
    2.Department of Computer Science and Engineering,University of South Carolina,Columbia,SC 29201,USA)
     
  • Received:2019-09-26 Revised:2019-10-20 Online:2020-03-25 Published:2020-03-25

摘要:

针对手机拍摄过程中产生的视频抖动问题,提出了一种基于光流法和卡尔曼滤波的视频稳像算法。首先通过光流法预稳定抖动视频,对其生成的预稳定视频帧进行Shi-Tomasi角点检测,并采用LK算法跟踪角点,再利用RANSAC算法估计相邻帧间的仿射变换矩阵,由此计算得出原始相机路径;然后通过卡尔曼滤波器优化平滑相机路径,得到平滑相机路径;最后由原始相机路径与平滑路径的关系,计算相邻帧间的补偿矩阵,再利用补偿矩阵对视频帧逐一进行几何变换,由此得到稳定的视频输出。实验表明,该算法在处理6大类抖动视频时均有较好的效果,其中稳像后视频的PSNR值相比原始视频的PSNR值约提升了6.631 dB,视频帧间的结构相似性SSIM约提升了40%,平均曲率值约提升了8.3%。

 

 

 

关键词: 视频稳像, 预稳定, Shi-Tomasi角点, LK算法, RANSAC算法, 卡尔曼滤波

Abstract:

Aiming at the video jitter problem caused by mobile phone shooting, a video stabilization algorithm based on optical flow method and Kalman filter is proposed. Firstly, the optical video method is used to pre-stabilize the dithered video, and Shi-Tomasi corner detection is performed on the pre-stabilized video frame. The LK algorithm is used to track the corner points, and then the RANSAC algorithm is used to estimate the affine transformation matrix between adjacent frames, so as to obtain the original camera path. Secondly, the smoothed camera path is optimized by the Kalman filter to obtain a smooth camera path. Finally, the relationship between the original camera path and the smooth path is used to calculate the compensation matrix between adjacent frames, and then the compensation matrix is used to geometrically transform the frames one by one, resulting in a stable video output. Experiments show that the proposed algorithm has good effects in processing six types of jitter videos. The PSNR value after image stabilization is increased by 6.631 dB compared with the original video, and the structural similarity (SSIM) between video frames is increased by 40%. The average curvature value is increased by about 8.3%.

 

Key words: video stabilization, pre-stability, Shi-Tomasi corner point, LK algorithm, RANSAC algorithm, Kalman filter