Chest X ray is the preferred choice for patients’ chest examinations and plays an important role in the diagnosis and treatment of patients. Doctors write chest X-ray diagnostic reports based on their own experience and habits. For some subjective or objective reasons, they will issue some abnormal diagnostic reports that do not match the diagnostic conclusions. Therefore, it is of great significance to carry out abnormal detection of the diagnostic reports. Chest X-ray diagnostic reports have many unknown words and sparse high-dimensional data and lack of a lot of effective labeling. Traditional methods are ineffective in detecting abnormal chest X-ray diagnostic reports. Therefore, this paper proposes an abnormal chest X-ray diagnostic report detection method based on topic model. Firstly, the bidirectional LSTM-CRF model is used to combine the character-level features in the chest radiograph diagnosis reports to obtain the specific medical terminology features, so as to solve the problem that the diagnosis reports have many unknown words and are described freely. Secondly, based on domain knowledge and template, the chest X-ray diagnosis reports are extended effectively to alleviate the problem of data sparsity. Finally, the LDA model is used to determine whether the image description in the diagnosis reports match the characteristics of the diagnosis conclusion, so as to detect the abnormal chest X-ray diagnosis reports. Experiments show that the accuracy of abnormal detection is 92.82 and the recall rate is 69.54 when the threshold is 2. The proposal has higher abnormal detection performance than the traditional methods.