We propose a new differential evolution (DE) algorithm which incorporates the DE with the crossover thought of the simplified swarm optimization (SSO) algorithm and the covariance matrix learning, called SCDE algorithm. As we know, mutation operation plays a very important role in the performance of the DE. However, traditional mutation strategies of the DE all use the relative position to generate a candidate vector. We try to utilize individual historical optimal solution to induce the variation, which can greatly enhance its ability of jumping out of local populations. Furthermore, the crossover and mutation operations are executed in an appropriate coordinate system which is generated by the covariance matrix of the population and it can make the crossover and mutation operation rotationally invariant. Experimental results show that the proposed algorithm can significantly improve the DE's performance on a set of 28 test functions in CEC 2013 benchmark sets.