Aiming at the prominent problems of ignoring the unnatural connection relationship and interaction relationship between human bodies in two-person interaction recognition algorithm, a two-person interaction recognition network based on improved spatial temporal graph convolutional model was proposed. Firstly, the edge features of joint point data were aggregated by edge convolution to capture the unnatural connectivity relations inherent in the human body. Secondly, the interaction relationship graph between two people was constructed by using the improved relationship network. Furthermore, the branch of edge convolution and the interaction relationship graph were embedded into the spatial temporal graph convolutional network block, which were constructed as an edge-graph convolutional block and interaction relation graph convolutional block. Finally, an improved spatial temporal graph convolution algorithm was proposed to capture both the unnatural connection relationship and the interaction relationship, so as to realized the recognition of two-person interaction behavior. To verify the effectiveness of the network, it was tested on the international public large-scale standard dataset NTU RGB + D. The experimental results show that the network obtain a recognition accuracy of 97.77%, which is an improvement of 4.28 percentage points compared to the baseline spatial temporal graph convolutional network. It improves the expressiveness of two-person interaction behavioral features, and achieves a better recognition effect than the existing state-of-the-art network models.