In order to improve the automation of convolutional neural network (CNN) design and further improve the accuracy and speed of prohibited item detection in complex background, a prohibited item detection algorithm for X-ray images was proposed based on neural network architecture search. First, a layer-by-layer progressive search strategy and a multi branch search space were designed, and the best side branches were searched for each layer structure based on batch normalization (BN) metric. Then, the backbone component of the target detection model was constructed based on the layer-by-layer progressive search strategy. Finally, a new data-driven X-ray image prohibited item detection model was formed. The experimental results have demonstrated that the algorithm achieves detection accuracy of 83.4%, 87.2% and 70.4% respectively on three datasets HiXray, OPIXray and PIDray. The algorithm proposed in this paper can adapt the dataset and automatically search for Backbone components with better performance from the dataset, and effectively improves the accuracy and speed of prohibited item detection compared with mainstream algorithms such as FCOS and YOLOv4.