We present a novel  abnormal behavior detection method  based on trajectory segment and topic model. In order to solve the problem of trajectory discontinuity caused by track deviation, all trajectories are firstly clustered by the fuzzy clustering algorithm, and then the sampling points of each segment of trajectory class are clustered by the latent Dirichlet allocation (LDA) topic model. The point of maximum probability is used as the visual word, and each trajectory class is represented by a series of visual words. In this sense, the local hidden Markov model (HMM) is established between every two visual words to detect abnormal trajectories through the  matching path method. Experimental results show that the proposed method can identify a variety of abnormal behavior, and improve the accuracy of abnormal behavior detection in CAVIAR database.