Abstract:

Stream computing is an important part of  big data computing, which has become a hot topic in big data research. Task management is one of the essential features of stream computing, and is responsible for resource scheduling and lifecycle management of stream computing tasks. Current researches focus on application requirements, architecture and overall technology of stream computing, and they are lack of dedicated investigation and analysis of task management techniques. Firstly, we present a general abstract function model of task management for stream computing systems. Secondly, we classify and analyze the key techniques for task management based on this model. Finally, we investigate their applications in  current stream processing systems, and the integration and optimization of above techniques.

Key words: big data stream computing, task management, abstract function model, resource allocation, data distribution, fault tolerance