Aiming at the video jitter problem caused by mobile phone shooting, a video stabilization algorithm based on optical flow method and Kalman filter is proposed. Firstly, the optical video method is used to pre-stabilize the dithered video, and Shi-Tomasi corner detection is performed on the pre-stabilized video frame. The LK algorithm is used to track the corner points, and then the RANSAC algorithm is used to estimate the affine transformation matrix between adjacent frames, so as to obtain the original camera path. Secondly, the smoothed camera path is optimized by the Kalman filter to obtain a smooth camera path. Finally, the relationship between the original camera path and the smooth path is used to calculate the compensation matrix between adjacent frames, and then the compensation matrix is used to geometrically transform the frames one by one, resulting in a stable video output. Experiments show that the proposed algorithm has good effects in processing six types of jitter videos. The PSNR value after image stabilization is increased by 6.631 dB compared with the original video, and the structural similarity (SSIM) between video frames is increased by 40%. The average curvature value is increased by about 8.3%.