关键词: Spark Streaming, CPU时间占用率, 分布式, 视频/图像处理

Abstract:

Intelligent video surveillance technology has a promising application prospect and growing demand in public safety, traffic management, smart city, etc. With a growing number of cameras used in video surveillance, the amount of image data collected by cameras is becoming bigger and bigger, which is out of the processing capacity of one single machine. The rise and development of distributed computing provides a good way to solve the problem of big data processing. We introduce a testing platform based on Spark Streaming, which is used to process video/image data received as stream, and illustrate the composition and working process of the platform. The impact of several important parameters on the performance of the cluster is deeply studied. In particular, the timeoccupancyrate of the CPU is initially proposed as one of the performance evaluating indicators, and together with the total processing time, it demonstrates the performance and resource usage of clusters more comprehensively.

Key words: spark streaming;time-occupancy-rate of CPU;distributed;video/image processing