Abstract: To solve the problem that the existing unsupervised video summarization algorithms do not accurately judge the importance of video frames, an unsupervised video summarization algorithm based on deep and shallow feature fusion is proposed. The deep features of video frames are extracted by a Convolutional Neural Network (CNN), while the shallow features are first extracted by the Speeded Up Robust Features (SURF) operator and then encoded using the Bag-of-Words (BOW) model. The deep and shallow features are fused to enrich the information of the feature descriptors as the input of the network model. A Bidirectional Long Short-Term Memory network (BiLSTM) is used to model the temporal information and output frame importance scores. The model is optimized using reinforcement learning. For generating static video summaries, a keyframe selection method based on local maxima is designed, which follows the temporal structure of the original video and avoids redundancy. Compared with several unsupervised video summarization algorithms on the SumMe and TVSum datasets, experimental results show that the proposed algorithm can make more accurate judgments on video content and generate higher-quality summaries.

Key words: video summarization, feature fusion, bi-directional long short-term memory (BiLSTM) network, reinforcement learning, local maximum