• 中国计算机学会会刊
  • 中国科技核心期刊
  • 中文核心期刊

Computer Engineering & Science

Previous Articles     Next Articles

Selecting distance metrics for incremental
clustering algorithm of high dimensional data

SHAO Junjian,WANG Shitong   

  1. (School of Digital Media,Jiangnan University,Wuxi 214122,China)
  • Received:2017-10-23 Revised:2018-06-20 Online:2019-02-25 Published:2019-02-25

Abstract:

Appropriate distance metric functions have an important effect on clustering results. For large-scale and high-dimensional datasets, the incremental fuzzy clustering algorithm is used to analyze the selection of distance metrics. Since the SpFCM algorithm divides a large-scale dataset into small samples for incremental batch clustering, it can get better clustering results in limited computer memory. Different distance metric functions are applied into the traditional SpFCM algorithm in order to measure the similarities between different samples to check the effect of different distance metrics on the SpFCM algorithm. Four distance metrics, which are the Euclidean metric, the cosine metric, the correlation distance metric and the extended Jaccard similarity metric, are used to calculate the distance for different large-scale high dimensional datasets. Experimental results show that, the latter three distance metrics can greatly improve the clustering effect. The correlation distance metric gets a better clustering result while the cosine distance metric and the extended Jaccard similarity distance get an average result.
 

Key words: high dimensional data, SpFCM algorithm, distance metric, incremental fuzzy clustering algorithm, correlation coefficient distance metric