With strong parallel computation power, GPU-based data analytic systems can achieve better performance than traditional CPU-based data analytic systems. However, how to leverage the resources of CPU and GPU to dispatch workloads appropriately in massive data scenarios remains to be solved. We propose a load balancing strategy on heterogeneous CPU-GPU data analytic systems. It adopts a pipeline model to split workloads, and a load balancing model based on pipeline is proposed to dispatch workloads to heterogeneous processors reasonably and appropriately, which reduces the total execution time of the system and enhances the performance. Experimental results show that the load balancing model based on pipeline  is suitable for various queries with different dataset volumes and has great performance.