• 中国计算机学会会刊
  • 中国科技核心期刊
  • 中文核心期刊

Computer Engineering & Science

Previous Articles     Next Articles

A rapid data processing method
for space science satellites

SUN Xiaojuan1,2,3,SHI  Tao2,3,LI Bing2,3,YANG Xiaoyan2,3,LEI Bin1,2,3,HU Yuxin1,2,3   

  1. (1.University of Chinese Academy of Sciences,Beijing 100049;
    2.Institute of Electronics,Chinese Academy of Sciences,Beijing 100190;
    3.Key Laboratory of Technology in Geospatial Information Processing and Application System,
    Chinese Academy of Sciences,Beijing 100190,China)
  • Received:2018-01-11 Revised:2018-03-20 Online:2018-08-25 Published:2018-08-25

Abstract:

Rapid data processing for largescale data acquired by satellites has always been the key to build spatial information processing systems. Faced with space science satellite problems such as allweather observation, multiple types of detection loads, and various processing algorithms, the existing data analysis methods based on CCSDS
 standard format are difficult to satisfy the current requirements of data processing systems for the onorbit space science satellites in correctness and timeliness. According to the observing data characteristics of the space science satellites, the paper proposes a fast data processing method for space science, designs the twolayer joint index structure that transforms the processing problem from the space science large data into the index tables and the source packet data units, and improves the efficiency of data processing. The paper designs a processing framework using scientific workflow technique, which supports the cooperation between the datadriven processing and the businessdriven processing of space science satellite data, and also supports various data processing workflows and the parallel scheduling of numerous tasks for different satellite payload types. The experimental results show that this method has good scalability and less memory usage, which has been well applied to the spatial information processing system of space science satellites.
 

Key words: data processing, space science satellite, index structure, scientific workflow