Abstract:

Ensemble pruning is an active research direction in the machine learning field. Ensemble pruning is an NPhard problem, most researchers use heuristics to obtain near optimal solutions. There are already many ensemble pruning approaches in literatures, but because of the different perspectives on which those methods are based, it is difficult to understand them clearly. In this paper, the ensemble pruning approaches are divided into four categories according to their pruning strategies: optimizationbased, rankingbased, clustering based and pattern miningbased. Next, the popular algorithms of each category are implemented and tested on 20 datasets from the UCI repository, and compared from three facets: prediction performance, pruning time and the size of the final ensembles. The advantages and disadvantages of each category are analyzed. The paper ends with some conclusions and future work.

Key words: ensemble learning;ensemble pruning;optimization based pruning;ranking based pruning;clustering based pruning;pattern mining based pruning