Traditional gesture interaction requires special interaction equipments such as Leap Motion or Kinect. In this paper, image channel conversion, binarization and other image processing methods are used to extract gestures. The planar motion information of gestures is obtained by the change of the coordinate value of the hand gestures. The problem of gesture depth is solved by the change of gesture area.The matching rate of different images is calculated by drawing the hand contour and using the image matching algorithm, and the highest matching rate is used to selecting the corresponding hand gesture motion information. The transformation matrix is calculated by the transformation from the camera coordinate system to the 3D scene coordinate system and the geometric transformation of 3D graphics, thus achieving the spatial movement and rotation of hand. Finally, the dynamic hand gesture interaction is realized without any specific gesture interaction devices.