Abstract: The development of medical imaging technology has generated a massive amount of medical image data, which reflects the internal structural features of the human body. Medical image segmentation technology can improve the efficiency of medical diagnosis, making it an important assistive tool for modern medical diagnosis. However, noise or artifacts that are inevitably present in the imaging process bring great challenges to the segmentation work. In existing segmentation models, single-frame medical image semantic segmentation models do not consider the relationship between image frames, while video semantic segmentation models utilize temporal information but have some limitations in edge extraction. To address these issues, this paper proposes a U-Net-based temporal semantic segmentation model modified by optical flow. This model can extract optical flow information between consecutive frames and perform feature extraction and weight allocation on the current frame and optical flow for correction. Experiments show that the model obtains optimal results on three evaluation metrics, namely Dice similarity, pixel accuracy and cross-merge ratio, on different types of datasets, namely Drosophila electron micrographs, combined healthy abdominal organ segmentation and coronary angiogram, which validate the effectiveness and generalization of the proposed model.

Key words: U-Net, optical flow, medical image, semantic segmentation, deep learning