Abstract:

Aiming at the following three problems of the kmeans algorithm:excessive dependence on the initial clustering center, slow convergence speed and insufficient memory when dealing with huge amounts of data, we present a new hybrid clustering algorithm called superkmeans for large data sets. The algorithm combines the kmeans algorithm with the improved highdimensional data clustering algorithm based on the supernetwork. We run it on the Hadoop clusters after the MapReduce parallel processing, and an ideal effect of clustering is achieved. Experimental results show that the algorithm not only improves the convergence and the clustering accuracy but also has high speedup and scalability performance.

Key words: k-means;super network;frequent itemsets；hypergraph partitioning;MapReduce