To tackle with the high nuisance alarm rate in phase sensitive optical time domain reflectometer (Φ-OTDR) system, an algorithm for pattern recognition based on multi-domain functionalities and a vector particle cluster optimization machine (PSO-SVM) was proposed. Firstly, the characteristics of the time-domain differential signal were extracted, and the differential signal was decomposed by wavelet packet. By verifying the accuracy of event classification corresponding to different decomposition layers, the optimal decomposition layer was set to 6 layers to extract the energy characteristics of the signal. Then, based on the SVM classifier, the PSO algorithm was used to optimize the SVM settings to improve the accuracy of optical fiber vibration recognition. Finally, the proposed model reconnaissance algorithm was validated using the Φ-OTDR event dataset. Experimental results show that the suggested model recognition algorithm achieves a classification precision of 95.6% of vibratory events.