Currently, with the development of air traffic management, an increasing number of artificial intelligence technologies are being applied to the field of control. Among them, automatic speech recognition technology is used for tasks such as correcting control instructions and conducting consistency checks for improving flight safety and efficiency. In order to address the issue of inconsistent performance in automatic speech recognition (ASR) systems, a performance evaluation method tailored to air traffic control (ATC) speech recognition systems was proposed in this study. Three ASR systems were evaluated and analyzed using this method. Firstly, ATC speech data were collected and annotated according to certain proportions of control scenarios to establish a testing corpus for ATC speech recognition systems. Secondly, an evaluation index system for ATC speech recognition systems was designed, and the weights of the indices were calculated using the Analytic Hierarchy Process (AHP). Lastly, three ASR systems tailored to ATC were proposed and trained for evaluation and analysis. The results show that this evaluation method enables comprehensive assessment of ATC speech recognition systems and analysis of their performance across different control scenarios. Moreover, it facilitates the provision of performance improvement suggestions tailored to specific control scenarios. It is concluded that this method offers an intuitive approach to evaluating ATC speech recognition systems and holds promise for guiding future research endeavors.