In order to solve the misjudgment caused by poor fit when the open transformer method is used to detect the short circuit fault between the stator core laminations of the AC motor. In this paper, the influence of the gap between the core section of the open-ended transformer and the stator tooth top on the detection results is analyzed theoretically. It is proposed to install the insulation gasket material on the original core section. The tolerance of the internal magnetic flux of the open-ended transformer core after the insulation gasket to the poor fit in the measurement process is deduced and verified, and the induced electromotive force characteristics of the open-ended transformer are further obtained. Then, the two-dimensional electromagnetic simulation verification was carried out on the stator of QFSN-300-2 turbine generator, and the experimental verification was further completed on the stator core of a small three-phase asynchronous motor. The results show that the poor fit between the core section and the top surface of the stator tooth has a great influence on the detection results in the measurement process of the open transformer method, which may lead to misjudgment. Adding a certain thickness of insulation gasket to the core section of the open transformer can greatly reduce the influence of the gap and improve the anti-interference ability of the detection.