Abstract: With the rapid development of social media, multi-modal semantic trajectory prediction has become a new challenge. The dependency between trajectory points plays an important role in prediction, but there are the following challenges: Trajectory sequence contains multiple modal information (time, points of interest, and activity text), so the multiple dependencies containing time, space, and activity intention exist and are complex. It is difficult for existing methods to quantify these complex dependencies. To solve the above problems, a Self-Attention mechanism based Multi-modal Semantic Trajectory Prediction model (SAMSTP) is proposed. Firstly, SAMSTP embeds multi-modal features jointly, designs a self-attention mechanism combined with Position Encoding to calculate the feature similarity among trajectory points, automatically learns and quantifies complex dependencies weights, and resolves the long-term dependency. Finally, LSTM network is used to capture the temporal relationship, and mode normalization is designed to prevent the dependency distortion and accelerate the convergence speed. Experiments on real datasets show that SAMSTP is effective and superior to the latest methods.

Key words: multi-modal, semantic trajectory, self-attention, mode normalization