Feature extraction is a key part of  image recognition, and precise feature expression can generate more accurate classification. We improve the recognition rate of the singlestage computational structure by adopting soft threshold encoder and the orthogonalizing visual dictionary of the orthogonal matching pursuit (OMP) algorithm. Besides, we build a twostage computational structure which extracts images’ features and increases the recognition rate. Experiments demonstrate that adopting softthreshold encoder and the OMP algorithm can increase the ability of extracting features of the singlestage computational structure and enhance imagerecognition rate in bigsample datasets. The twostage computational structure can improve recognition rate on selfselection datasets. The OMP algorithm can improve recognition rate of the VOC2012 dataset. For selfselection datasets, the twostage computational structure outperforms the singlestage computational structure and network in network (NIN), and is equivalent to convolutional neural networks (CNN), indicating that the twostage computational structure is adaptive to selfselection datasets.