The philosophy of structural similarity is a new idea about image quality assessment, which models the low level composition of Human Visual Systems(HVS). The paper introduces the shortcomings of the traditional image quality assessment methods firstly,and analyzes the structural similarity theory based the human visual systems. As an implementation of the new philosophy, the Structural Similarity model(SSIM) is simple and has been proved to be better than the PSNR (Peak Signal to Noise Ratio) or the MSE (Mean Square Error)model, but there still remain some deficiencies in assessing badly blurred images. On the foundation of SSIM,the paper proposes a new image quality assessment method based on the Edge Structural Similarity(ESSIM). It makes use of the relationship between the edge of image and the human visual system. The experimental results show that the ESSIM model is an effective method, and more consistent than the SSIM model.