This paper proposes a multi-modal deep neural network grasping detection network, which is mainly aimed at the low accuracy of grasping and detecting unknown objects in the task of manipulator grasping detection. Firstly, the residual module is introduced into the RGB and depth channels, and the performance of the network is enhanced. Then, a multimodal feature fusion module is used to merge the feature. Finally, the fusion feature is returned to the full connection layer to get the best capture test results. The experiment results show that the precision of the proposed algorithm is 95.7% and the precision of object segmentation is 94.6% on Cornell dataset.In addition, this paper proves that the introduction of the residual module can improve the quality of network capture through ablation experiments.