The widespread dissemination of online rumors and their negative impact on society urgently require efficient rumor detection. Due to the lack of semantic information and strict syntactic structure in the text of the dataset, it is meaningful to combine user characteristics and contextual features to enrich semantic information. In this regard, MRUAMF is proposed. Firstly, four indicators including user information completeness, user activity, user communication span, and user platform authentication index are extracted to construct a quantitative calculation model for user authority. By cascading user authority and its constituent indicators, and using a two-layer fully connected network to fuse features, user characteristics are effectively quantified. Secondly, considering the effectiveness of context in understanding rumors, relevant contextual features are extracted. Finally, the BERT pre-training model is used to extract text features, which are then combined with the Multimodal Adaptation Gate (MAG) to fuse user features, contextual features, and text features. Experiments on the microblog dataset show that compared with the baseline model, the MRUAMF model has better detection performance with an accuracy rate of 0.941.