• 中国计算机学会会刊
  • 中国科技核心期刊
  • 中文核心期刊

Computer Engineering & Science ›› 2025, Vol. 47 ›› Issue (4): 706-717.

• Artificial Intelligence and Data Mining • Previous Articles     Next Articles

BigFlow: A service system for cross-center collaborative analysis of scientific data

ZHU Xiaojie1,2,CHENG Zhenjing1,WANG Huajin1,YANG Gang1,TIAN Yao1,FAN Dongwei3,MI Linying3,LIANG Zhaoji1,2   

  1. (1.Computer Network Information Center,Chinese Academy of Sciences,Beijing 100083;
    2.University of Chinese Academy of Sciences,Beijing 100049;
    3.National Astronomical Observatories,Chinese Academy of Sciences,Beijing 100101,China)
  • Received:2024-07-04 Revised:2024-08-23 Online:2025-04-25 Published:2025-04-17

Abstract: The integration of big data technology and scientific data has spawned numerous new paradigms for scientific research and brought about a widespread need for cross-center collaborative analysis of scientific data. However, such analysis faces significant technical challenges, including inefficient cross-center data transfer, difficulties in cross-framework heterogeneous computing, and low efficiency in cross-center job scheduling, while also requiring trustworthiness throughout the analysis process. To address these technological challenges, a scientific data cross-center collaborative analysis service system called BigFlow has been developed.The systems cross-center collaborative analysis capabilities have been tested and validated based on scenarios such as large-scale astronomical star catalog cross-matching and the identification of check dam locations in the Yellow River basin.

Key words: integrated analysis, cross-center collaborative analysis, cross-framework workflow, trustworthy analysis