Semi-supervised support vector machine (S3VM) based on label mean can lead to low classification accuracy and unstable results due to random selection of unlabeled samples. In order to deal with the problems, we propose a semisupervised support vector machine based on clustering label mean. This method modifies the penalty terms of the original algorithm for unlabeled samples, clusters unlabeled samples and replaces label mean with clustering label mean. Experimental results indicate that the proposed method greatly reduces the misclassification of background and objectives, improves the stability and classification accuracy of the algorithm, and it is suitable for image classification.