A two-stage damage quantitative detection method based on difference method and improved particle swarm optimization (PSO) was proposed to solve the structural damage problems of laminates during machining and service. Firstly, the structural damage element was defined by the method of reducing the elastic modulus of the element. The modal parameters of the numerical model of laminates under different damage conditions were obtained by analyzing the structural vibration characteristics, and the approximate damage area was identified by the difference method. Secondly, the precise location and degree of damage are calculated quantitatively by using improved particle swarm optimization algorithm. The objective function was constructed by using the flexibility matrix of the modal parameters of the tested structural model and the constantly updated finite element model, and the variation factor was introduced to make the algorithm effectively escape from local convergence. MS operator greatly reduced the influence of the healthy element on the damage degree of the real damage element. The results show that the location and degree of damage can be accurately identified by the improved particle swarm optimization algorithm.