关键词: 系统执行效率, 作业日志分析, 用户行为, 作业失败预测, 机器学习

Abstract: The scale of supercomputers is expanding. Meanwhile, the complexity of scientific applications is also increasing, which leads to many job failures on supercomputers. These failed jobs causes a waste of resources and prolong the waiting time of queuing jobs, which seriously affects the reliability of the system. If these failed jobs can be predicted in advance, necessary measures can be taken to improve the system resource utilization and system execution efficiency, which is very important for the future exascale supercomputers. Therefore, this paper attempts to predict these job failures from the known traditional features and construction features, and find the features and processing methods that can reflect the users work behavior patterns and submission behavior patterns. By combining behavior features and traditional features, a comprehensive framework based on tree structure model is proposed to predict job failure. The prediction experimental results show that the comprehensive prediction framework is better than the single model prediction, and the comparative experimental results show that the prediction effect is better than other related methods.

Key words: system execution efficiency, job log analysis, user behavior, job failure prediction, machine learning