关键词: 数据处理, 故障识别, 混合异构, 语义分析, 潜在狄利克雷分布

Abstract: This paper presents a method for detecting job anomalies in large-scale distributed HPC heterogeneous platforms.Analyzing job runtime logs is vital for detecting anomalies,but the sheer volume of logs hinders human comprehension.To address this,we introduce a multi-source log semantic analysis approach using latent Dirichlet allocation (LDA) to analyze logs from various sources.By modeling topic evolution over time and matching with historical faulty job patterns,it predicts anomalies.Experiments on a domestic HPC platform show 95.2% precision,enhancing predictive capability and aiding users and administrators in quickly diagnosing issues,thereby improving HPC environment availability and efficiency.

Key words: data processing, fault identification, hybrid heterogeneity, semantic analysis, latent Drichlet allocation（LDA)