Classification of short texts is challenging due to their typical characteristics of severe sparseness and high dimension. We propose a novel approach to classify short texts by combining both lexical and semantic features. To construct the term dictionary, we firstly select lexical features of the most distinctive words of a certain category, and then extract the optimal topic distribution from the background knowledge repository based on the Latent Dirichlet Allocation so as to construct the new features of short texts. Experiments on classifying Sohu news titles which are typical short texts via SVM and K-NN show that our method can greatly improve the classification results.