关键词: Spark, 乘潮水位, 任务调度算法, 并行处理, 海洋环境信息

Abstract:

Tidebound water level computing is an important part of ocean environment information processing, which features huge amount of data, high complexity, and prolonged computing time. The traditional computing model implemented by HPC has a number of problems, such as high computation cost, poor scalability and interactivity. Aiming at all these problems, we propose an interactive computing and visualization platform based on the Spark scheduling algorithm. We design a computing capacity scheduling algorithm, realize the parallel processing of largescale tidebound water level data, such as data retrieval, data extraction, numerical calculation, featurebased visualization, and achieve the purpose of parallel processing and visualization of largescale ocean environmental data on Spark. Experimental results show that the computing and visualization platform based on Spark can improve the traditional computing model, lessen the dependence of tidal level calculation on high performance cluster and reduce computation cost. In addition, the newlydeveloped task scheduling algorithm can make task allocation more rational and scientific, and therefore further enhance its efficiency. In conclusion, the proposed platform provides a new method for tidebound water level computing.

Key words: Spark, tide-bound water level;task scheduling algorithm;parallel processing;ocean environmental information